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ABSTRACT 



As policymakers have sought to balance the goal of fostering 
poor children's well-being with that of encouraging adult's self-sufficiency, 
public assistance has become more predicated on custodial parents' 
involvement in work or mandatory welfare- to-work programs. This summary 
report examines the effects of welfare- to-work programs implemented as part 
of the Job Opportunities and Basic Skills Training Program (JOBS) on the 
children of adults mandated to participate. The evaluation focused on 
children's development and well-being in a sample of families with 
preschool-age children at the start of the evaluation. The findings indicated 
that the welfare- to-work programs implemented as part of the JOBS program did 
have significant impacts on children's development, but these impacts were 
not widespread and were generally small. There were favorable impacts 
particularly in the area of cognitive development and particularly at one 
site. There were unfavorable child health impacts, specifically at one site. 
Findings in the area of children's behavior and emotional adjustment 
encompassed both favorable and unfavorable impacts . Findings for a subgroup 
of higher risk families were few and tended generally to be small. 

Unfavorable impacts were found for children from lower-risk families at three 
sites. The unfavorable and policy relevant impacts occurred in a concentrated 
manner and some had moderate to large effect sizes. It was concluded that 
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most children were not adversely affected, and in some cases were helped, by 
their mothers' participation in a JOBS program. Further follow-up will be 
important because of the limited unfavorable impacts. (Contains 37 
references.) (KB) 
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SUMMARY REPORT 



THE NATIONAL EVALUATION OF 
WELFARE-TO-WORK STRATEGIES: 

IMPACTS ON YOUNG CHILDREN AND THEIR FAMILIES 
TWO YEARS AFTER ENROLLMENT: 

FINDINGS FROM THE CHILD OUTCOMES STUDY 



Martha J. Zaslow, Sharon M. McGroder, and Kristin A. Moore 



I. Overview 



As we seek to understand the effects on families of the 1996 welfare reform law, we can 
build on the foundation of a rigorous evaluation study focusing on the effects on families of 
welfare-to-work programs implemented under the previous welfare law, the Family Support Act 
of 1988. The National Evaluation ofWelfare-to-Work Strategies (the NEWWS, funded by the 
U.S. Department of Health and Human Services and the U.S. Department of Education) is 
evaluating the impact of a set of welfare-to-work programs operated under JOBS (the Job 
Opportunities and Basic Skills Training Program). A pioneering feature of this national 
evaluation is that it simultaneously considers program impacts on adult economic outcomes and 
on the development and well-being of the children in the families. 

This summary report presents a summary of findings from one of a set of three 
complementary reports from the National Evaluation ofWelfare-to-Work Strategies ’ two-year 
follow-up (with results from a further follow-up, completed five years after families enrolled, to 
be presented in the future). We focus in the present summary report on the findings related to 
impacts on children . reporting results from a special component of the evaluation, the Child 
Outcomes Study (see McGroder, Zaslow, Moore and LeMenestrel, 2000, for a detailed 
presentation of findings). This component of the evaluation focuses in depth on children s 
development and well-being for a sample of families with young (preschool-age) children at the 
start of the evaluation, drawn from three of the evaluation ’s seven research sites. A second 
report in this series focuses primarily on economic impacts in all seven of the evaluation s 
research sites, with a more limited examination of impacts on children of all ages (Freedman, 
Friedlander, Hamilton, Rock, Mitchell, Nudelman, Schweder and Storto, 2000). A third report 
draws together findings on children from the in-depth look at young children in the Child 
Outcomes Study, and brief markers of well-being collected regarding children of all ages in 
families in the full evaluation sample (Hamilton, with Freedman and McGroder, 2000). 
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The Child Outcomes Study of the Notional Evaluation of Welfare-to-Work Strategies 
examines the impacts on both the parental and child generations of two distinct approaches to 
welfare reform implemented as part of the federal JOBS Program: a labor force attachment 
approach (emphasizing a rapid transition to employment), and a human capital development 
approach (emphasizing a longer-term strategy of education and training in order to obtain a 
better job). These strategies are precursors of the welfare reform programs now being 
implemented under the 1996 welfare law, the Personal Responsibility and Work Opportunity 
Reconciliation Act (PRWORA). The labor force attachment approach under JOBS is especially 
gei mane because of its emphasis on moving clients quickly into employment, the clear priority of 
new policies. However, the human capital development approach may provide an informative 
model for states as caseloads drop, and those families remaining on welfare face more barriers 
to employment (such as low literacy or limited education). 

Although welfare policies were initiated many years ago with the aim of protecting 
children in poor families, most of the evaluation research concerning these policies has focused 
on adult economic outcomes. This is perhaps not surprising, given that the most clearly targeted 
outcomes of these programs have been economic. The Family Support Act explicitly stated as its 
goal the reduction of long-term welfare dependency. Further, this law did not call for services 
aimed directly at enhancing the development of children (such as early childhood educational 
intervention, or developmental screening); rather authorized services focused on increasing 
adult employment. 

Nevertheless, a mother’s assignment to a welfare-to-work program has the potential to 
affect the development of children, for example, by affecting the material resources available 
within the family, and by affecting children ’s experiences of care both within and outside of the 
home. The Child Outcomes Study of the National Evaluation ofWelfare-to-Work Strategies 
examines whether children can be affected by their mothers ’ assignment to a welfare-to-work 
program, how their development and well-being are affected (favorably or unfavorably), if at all, 
and in whaj wavs any impacts on children come about. 

The Child Outcomes Study uses a rigorous experimental design. Two years after mothers 
were randomly assigned to one of the two JOBS welfare-to-work strategies or to a control group, 
outcomes for children (at that point between about 5 and 7 years of age) were examined. The 
children s cognitive development and academic achievement were measured through a 
combination of direct assessment (an assessment of the children 's cognitive school readiness) 
and maternal report (for example, mothers ’ reports of academic problems). The children ’s 
behavioral and emotional adjustment were measured through maternal report (for example, 
using measures of the child’s behavior problems and positive social behaviors). The children ’s 
health and safety were also measured through maternal report (for example, using an interview 
measure indicating whether the child has had an accident, injury or poisoning requiring 
emergency medical attention; an interview measure widely used in national surveys in which the 
mother indicates whether she sees the child ’s overall health as excellent, very good, good, fair, 
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or poor). Validation work indicates that the global health rating reflects primarily physical 
health problems (Krause and Jay, 1994). In general, all measures selected for use in the Child 
Outcomes Study have demonstrated sufficient validity and reliability (Bracken, 1984; Polit, 

1996; Zill, 1985; Peterson and Zill, 1986). 

In addition to examining mean scores on measures of cognitive school readiness, 
problem and positive behavior, and overall health, we also examined program impacts on the 
proportion of children with extreme scores on these measures in the interest of ascertaining 
whether JOBS welfare-to-work programs changed the distribution of children ’s outcomes --for 
example, reducing the proportion at the "unfavorable end and/or increasing the proportion at 
the "favorable" end -- which is possible even if the programs had no impacts on mean scores. 
Thus, in some cases a single response to a survey question can give rise to two or more impacts 
(e.g., one relating to the mean and one relating to the distribution). 

The Child Outcomes Study was conducted in three sites: Atlanta, Georgia; Grand 
Rapids, Michigan; and Riverside, California. Families enrolled in the evaluation between 
September 1991 and January 1994. Data collection for the two-year follow-up was completed in 
January 1996, prior to the implementation of the new welfare law. Findings from this study 
must be seen in light of the fact that mothers were exempt from participation in JOBS welfare-to- 
work activities if they were needed at home to care for an ill or incapacitated family member, 
including a child. As a result, children with a health condition requiring such care were not 
included in the evaluation. The 1996 welfare law no longer provides an explicit exemption for a 
mother with an ill or incapacitated child. 

In this experimental evaluation, mothers randomly assigned to a control group in each of 
the study sites were eligible for all welfare benefits. However , they did not receive the special 
messages and case management of a JOBS program, they were not mandated to participate in 
JOBS program activities, and they did not have access to the particular work preparation 
activities through JOBS. Control group members were eligible for child care assistance, similar 
to that offered to program group members, if they were participating in work preparation 
activities in which they had enrolled on their own. 

The summary report that follows this overview and the full report of the two-year follow- 
up of the Child Outcomes Study provide extensive information on the impacts of the six JOBS 
programs examined here (i.e., the human capital development and labor force attachment 
approaches in each of the three study sites) on economic outcomes in these families with young 
children, as well as on such further measures of family functioning as marital status, father 
involvement as reported by mothers, maternal psychological well-being and parenting. In this 
overview, we focus specifically on the findings regarding impacts on children. 

Overall, the six JOBS programs examined had relatively few statistically significant 
impacts on children on average, that is, across all families assigned to a given program. 
Nevertheless, there were more impacts than one would expect due to chance. In addition, 
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findings differed according to the aspect of the children 's development examined, with impacts in 
the area of cognitive development favorable, in the area of health unfavorable, and in the area of 
behavior mixed (including both favorable and unfavorable impacts). 

Child impact findings at the aggregate level tended to differ according to site. Whereas 
impacts on child outcomes were relatively few, they tended to be favorable in Atlanta, 
unfavorable in Riverside, and mixed in Grand Rapids. Where impacts were found, they were 
generally small. Only a single impact' was of sufficient magnitude to be called "policy relevant. " 
P olicy relevant findings in the present study are those that meet the criterion, set at the start of 
the study, of having an effect size of at least a third of a standard deviation . 2 The strongest 
evidence on which to base conclusions about impacts on children is a consistent patterning of 
impact results, particularly when impacts meet or exceed the criterion for policy relevance. 

In addition to examining impacts for the sample as a whole in each site, analyses also 
addressed the question of whether findings differed for children from families with different 
background characteristics. There was an initial concern that children in families at higher risk 
(in terms of such background characteristics as the mother ’s education, her work history, 
number and spacing of children, and maternal psychological well-being) might show a pattern of 
unfavorable program impacts. Mothers in higher-risk families might be less able to fulfill the 
requirement to participate in a welfare-to-work program, or might experience substantial stress 
in doing so. Children in higher-risk families might be more vulnerable to changes in family 
routines and their own care situations. Yet on the other hand, JOBS programs might provide 
exactly the kind of support the higher-risk families need in order make progress toward 
economic self-sufficiency. The combination of messages, enhanced case management, program 
requirements, and services might be particularly effective for such families. If this is indeed the 
case, then children in higher-risk families might be particularly likely to show favorable impacts 
of one or both JOBS program approaches. 

As for the findings for children across all families, statistically significant impacts for 
subgroups of families were relatively few, though exceeding what might have been expected on 
the basis of chance. Findings for children in higher-risk families, when they emerged, tended to 
be favorable, though small, when the mother had been assigned to a human capital development 
program or to Atlanta’s labor force attachment program 3 . By contrast, program impacts, some 



] an unfavorable impact of Riverside 's labor force attachment program on the percent of focal children 
rated by their mothers to be in very good or excellent health . 

All statistically significant impacts on measures of children ’s development warrant continued monitoring. 
However, impacts that meet the criterion for policy relevance are relatively larger impacts and, thus, may have 
greater implications for children 's development. Consequently, policy makers may want to take special note of 
these impacts in their policy decisions. 

3 The only exceptions relate to unfavorable health impacts of Riverside’s human capital development 
program, a single unfavorable health impact of Atlanta ’s labor force attachment program, and a single unfavorable 
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of which peached the criterion for policy relevance, tended to be unfavoi able for children from 
higher-risk families in the other two labor force attachment programs. 

Contrary to the initial hypothesis , it was among lower-risk families that there were 
indications of some concentrations of unfavorable impacts for children, and these did not appear 
to vary by program approach. Rather, a pattern of unfavorable impacts occurred for children in 
three particular programs: Grand Rapids labor force attachment program, and both of 
Riverside 's programs. In addition, a number of the program impacts for children in lower-risk 
families met the criterion for policy relevance, and nearly all of these were unfavorable impacts. 
Further, many (one-third) of the unfavorable and policy relevant impacts found for children in 
lower-risk families substantially exceeded the threshold for policy relevance, in that effect sizes 
were .50 or larger (which is considered "moderate" to "large" in magnitude: Cohen, 1 988). 

Further analyses were conducted to begin to explicate the processes underlying program 
impacts on children. While the analyses of program impacts on children summarized above are 
experimental in nature, the analyses used regarding the processes underlying program impacts 
on children, mediational analyses, are non-experimental. Whereas experimental analyses 
provide strong evidence regarding the existence of program impacts on children, they cannot 
address the question of how these impacts came about. Because mediational analyses are non- 
experimental, they do not allow firm causal inferences to be made regarding the pathways 
through which these impacts come about. Nevertheless, if one hopes to gain insight into the 
processes through which a given program had impacts, then non-experimental statistical 
analyses are necessary. 

Mediational analyses examined five selected program impacts on children in greater 
depth. The particular child impact findings focused upon in this way were selected in keeping 
with the overall patterning of findings of favorable impacts for children in the area of cognitive 
development, unfavorable health impacts, and mixed behavioral impacts. These analyses rely on 
information about processes and child outcomes measured contemporaneously and used a 
relatively modest statistical approach in modeling pathways; as such, results should be 
considered preliminary. Additional waves of data from the five-year follow-up, as well as 
statistical methods that more effectively partition effects among multiple mediators, more 
completely control for selection effects, and allow alternative models to be tested explicitly (e.g., 
structural equation modeling), are needed to provide more definitive answers regarding the 
multiple pathways through which program impacts on children came about. 

These analyses suggest that the mechanisms through which children can be affected by a 
given welfare-to-work program include both outcomes that were directly targeted (e.g., 
employment), as well as outcomes not directly targeted by the programs (e.g., parenting, 
maternal psychological well-being). In particular, though, the findings highlight the role played 



behcn’ioral impact of each of Atlanta ’s JOBS programs. 
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by mechanisms more proximal to the child, especially maternal psychological well-being and 
parenting. As yet, the roles of child care, health insurance, and income in mediating the five 
child impacts selected for pathways analyses have not been identified, although it should not be 
concluded that these factors do not in general affect outcomes for children. Findings also 
indicate that welfare-to-work programs sometimes have effects on aspects of family life that are 
important to children that go in opposing directions (favorable and unfavorable). For example, 
Atlanta 's labor force attachment program increased mothers ’ sense of time stress (which was 
associated with less favorable behavioral outcomes in children), but simultaneously led to 
improvements in parenting ( which was associated with more favorable behavioral outcomes in 
children). Impacts on children reflect the net effect of such influences. Opposing influences of 
this kind on key aspects of family life may help explain the small number and size of significant 
impacts on child outcomes. Understanding the nature and direction of multiple influences on 
family life is central to strengthening pathways that yield favorable impacts on children and 
weakening the pathways that have detrimental implications for children. 

In sum, the results indicate that the welfare-to-work programs implemented as part of the 
.JOBS Program did have significant impacts on children ’s developmental outcomes, but these 
impacts were not M’idespread and were generally small. When impacts did occur, they were 
favorable in the area of the children s cognitive development and academic achievement, 
unfavorable in the area of the children ’s health and safety, and mixed in the area of behavioral 
and emotional adjustment. Looking at subgroups of families, there was a pattern of favorable 
impacts for children from higher-risk families assigned to human capital development programs 
or to the Atlanta labor force attachment program. Yet, at the same time, there was a 
concentration of unfavorable, and policy relevant, impacts for children from lower-risk families 
in three of the programs studied. 

Given these findings, we must consider it a possibility that the welfare-to-work programs 
implemented under the 1996 law can have impacts — both favorable and unfavorable — on 
children ’s development. It is indeed possible that the intensified obligations and opportunities of 
the new policy are resulting in stronger or more pervasive impacts on children than were found 
for the JOBS Program. Furthermore, the variations in impacts suggest the value of research 
that would examine other policy strategies, for example, childcare, health care, and other work 
supports, to determine how to promote favorable, and avoid unfavorable, child impacts. Thus, it 
is important to continue to measure the influences of welfare-to-work programs and other policy 
strategies on both the parent and child generations. 



-vi- 





References 



Bracken, B. A. (1984). Bracken Basic Conceal Scale: Examiner ’s Manual . The 
Psychological Corporation, Har court Brace Jovanovich, Inc. 

Cohen, J. (1988). Statistical power analysis for the behayionil sciences (2nd edition) . 
Hillsdale, NJ: Erlbaum and Associates. 



Freedman , S., Friedlander, D., Hamilton, G., Rock, J., Mitchell, M., Nudelman, J., 
Schweder, A., andStorto, L. (2000). Evaluating alternative welfgr^tQz^ojk ap proaches : Two- 
year impacts. for eleven programs . Washington, D.C..' U.S. Department of Health and Human 
Services, Administration for Children and Families and Office of the Assistant Secretary for 
Planning and Evaluation, and U.S. Department of Education. 

Hamilton, G„ with Freedman, S. and McGroder, S. (2000). Do mandato ry, welfare-to- 
work programs effect, the welfbeing of children 7_ A synthesis of child_ re search conducted_ as_ 
part ofthe National Evaluation of Welfare -to- Work Strategies. Washington, D.C.: U.S. 
Department of Health and Human Services, Administration for Children and Families and Office 
ofthe Assistant Secretary for Planning and Evaluation, and U.S. Department of Education. 

Krause, N. M. and Jay, G.M. (1994). What do global self-rated health items measure?' 
Medical Care , 32 (9), 930-942. 

McGroder, S.M., Zaslow, M.J., Moore, K.A., and LeMenestrel, S.M. (2000). The 
National Evaluation ofWelfare-to-Work Strategies^ Impacts on young children and their 
families two years_ after enrollment: Findings from the Child_ Outcomes Study. Washington, 

D. C. : U.S. Department of Health and Human Services, Administration for Children and 
Families and Office of the Assistant Secretary for Planning and Evaluation, and U.S. 

Department of Education. 

Peterson, J., and Zill, N. (1986). Marital disruption and behavior problems in children. 
Journal of Marriage and_ Family , 48, 295-307. 

Polit, D. F. (1996). Self administered teacher questionnaire in the New Chance 42- 
month survey . New York: Manpower Demonstration Research Corporation. 



Zill, N. (1985). MmiQL Problem Index , Washington, D.C. : Child Trends, Inc. 



II. Goals of this Report 



this summary report highlights key findings from a study focusing on young children in 
the context of welfare-to-work programs: the two-year follow-up report of the Child Outcomes 
Study of the National Evaluation of Welfare-to-Work Strategies (McGroder, Zaslow, Moore and 
LeMenestrel, 2000). Specifically, this report asks whether young children’s development and 
well-being were affected when their mothers were assigned to participate in welfare-to-work 
programs implemented as part of the Job Opportunities and Basic Skills Training (JOBS) 
Program, the set of programs authorized by the Family Support Act of 1 988. 

The JOBS Program required mothers with children as young as age three (or age one, at 
state option) to participate in welfare-to-work activities. This was the first time that a federal 
program implemented nationally required participation by mothers with preschool-age children. 
Young children are particularly likely to experience changes in their daily lives when their 
mothers’ activities change. In addition, there is evidence that young children’s development is 
particularly sensitive to family poverty status (Duncan and Brooks-Gunn, 1997). Thus, it is 
important to consider the implications of welfare-to-work programs on children who are 
preschool-age when their mothers enroll in these programs. 

In this report we ask whether different aspects of young children’ s development 
(specifically their cognitive development and academic achievement, behavioral and emotional 
adjustment, and health and safety) were affected at all, and, if so, for better or for worse, two 
years after their mothers were assigned to participate in a JOBS welfare-to-work program. We 
also ask whether aspects of family life that are important to children’s development (for example, 
the stimulation and support available to the child in the home environment, participation in child 
care, and poverty status) changed when mothers were assigned to participate in a JOBS welfare- 
to-work program. Finally we explore the linkages between changes in family life and program 
impacts on children’s developmental outcomes. 

Why focus on outcomes fOL children lU evaluating a set of adult-oriented welfare-to-wo rk 
programs? 

Much of what is known about the effects of welfare-to-work programs focuses on 
economic outcomes for adults, for example, the extent to which programs increase employment, 
earnings and household income, and reduce welfare receipt. Yet a majority of those receiving 
public assistance are children. Further, a primary purpose of welfare programs has historically 
been to protect the well-being of children growing up in poor, and especially single-parent 
households. Even those programs that target adult economic outcomes, and have few or no 
components explicitly focused on enhancing young children’s well-being, have the potential to 
alter important aspects of children’s lives. For example, programs that target family economic 
self-sufficiency can alter the time children spend in the home and in child care settings, and the 
material resources available for their needs. Thus, even though JOBS programs did not aim 
directly to change child well-being (for example, through an early childhood intervention, or 
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through developmental screening and follow-up services), children’s development could have 
been affected indirectly , because of changes within the family and in children’s care situations. 

Accordingly, this study asks: (1) whether JOBS programs affected the well-being and 
development of children; (2) what the direction of effects on children was, that is, whether 
children’s development and well-being were influenced favorably or unfavorably; (3) which 
children were affected (all children, on average, or primarily children in certain types of 
families); and (4) how children were.affected by their mothers’ assignment to a JOBS program. 

This study uses a rigorous design to examine program impacts on children. The-sample 
for the Child Outcomes Study is a subset of the families that were participating in the larger 
evaluation of economic impacts of JOBS called the National Evaluation of Wei fare- to- Work 
Strategies. The subset of families in the Child Outcomes Study includes families in three of the 
full evaluation’s seven research sites, and specifically families that had a preschool-age child at 
the time of enrollment in the evaluation. 

The larger evaluation, and the Child Outcomes Study embedded within it, followed an 
experimental design. Families were randomly assigned to a control group, or to one of two 
experimental groups. The control group families were eligible for all benefits associated with Aid 
to Families with Dependent Children; however, they had no requirement to participate in a 
welfare-to-work program, nor did they receive the special messages, case management, or 
services of a JOBS program. The two experimental groups were also eligible for all benefits, but 
were required to participate in a JOBS welfare-to-work program. The two experimental groups 
differed in terms of which kind of program they were randomly assigned to, with one approach 
(labor force attachment) emphasizing a rapid transition to employment, and the other (human 
capital development) emphasizing education and training as a means to obtaining a better job. 
Families in the two experimental groups had the encouragement and support of enhanced case 
management. At the same time, they faced the possibility of sanctions (reductions in welfare 
benefits) if they did not participate in welfare-to-work activities. 

In an experimental study of this kind, we are confident that families did not differ 
systematically before they were randomly assigned to a control or experimental group (and this 
was confirmed statistically). Accordingly, differences between those in an experimental group as 
opposed to the control group can be attributed to the influence of having been assigned to a 
JOBS program. We use the term child outcome to refer to the measures of child well-being and 
development that we are focusing on. We reserve the term child impact specifically to describe 
differences on a child outcome measure between families in an experimental and control group. 

In addition to assessing impacts, we also look at the child outcome scores for children in 
the control group in order to describe the developmental status of children in the absence of a 
JOBS welfare-to-work program. In this way, we can examine the extent to which these children 
were at risk in their development apart from exposure to a welfare-to-work program. This 
provides an important context for understanding program impacts. 
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We make this distinction in terminology for measures of family well-being as well, with 
the term "outcomes" referring to the measures of family functioning in economic and other areas, 
but the term "impacts" referring to experimental and control group differences on these outcome 
measures. 

Did we anticipate effects on childrm to go in a particular direction? 

Very little is known about how welfare-to-work programs affect children, and the Child 
Outcomes Study is one of the first to focus on this issue. Given the very limited basis for making 
predictions, we did not begin the study anticipating that JOBS programs would affect children in 
a particular direction, with either improvement or deterioration in the children’s development. 
Instead, the study was initiated in order to explore four diverging possibilities: 

(1) That children’s development would be affected favorably when their mothers were 
assigned to a JOBS program . Mothers assigned to a JOBS program could potentially 
experience gains in educational attainment, basic skills, and job skills. Over time, 
participation could lead to employment, improvements in earnings and income. Through 
such changes, mothers could also experience increased psychological well-being, and the 
home environment could improve. Increased child care participation, given that the care 
was of good quality, could provide children with increased stimulation and support. Such 
changes could affect children’s development positively. 

(2) That children’s development would be affected unfavorably when their mothers were 
assigned to a JOBS program . This direction of impacts on children is possible if mothers 
experienced substantial stress in making a change in their daily activities and in arranging 
family life so that they could participate in welfare-to-work activities. Such stress could 
affect the quality of mothers’ interactions with their children and the degree of 
organization (for example of routines, the physical setting) of the home environment. 
Unfavorable impacts on children could occur if the young children were exposed to 
unstable (frequently changing) child care arrangements or child care of poor quality. 
Unfavorable impacts on children could also occur if mothers made the transition to 
employment, but did so without stable earnings, or with total family income falling below 
what they were receiving while on welfare. 

(3) That only children in certain subgroups of families would be affected. Families with 
particular characteristics might respond in different ways to the mothers’ assignment to 
JOBS programs. For example, mothers with limited education, literacy, and/or previous 
work experience, or mothers with symptoms of depression, might find it particularly 
difficult to fulfill welfare-to-work program requirements. Such mothers might fail to 
fulfill the requirements and face sanctioning. Alternatively, they might fulfill the 
requirements, but do so with substantial stress. Children in these families, but not others, 
might show unfavorable program impacts. On the other hand, there might be families 
especially likely to benefit from JOBS programs. For example, mothers with limited 
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education and literacy, mothers with limited work experience, or mothers experiencing 
psychological distress, might benefit especially from the combination of supports 
(enhanced case management), program requirements (a participation mandate), and 
program messages and services (labor force attachment and human capital development 
welfare-to-work activities) that comprised JOBS programs. If the mothers in such 
higher-risk families are particularly responsive to the JOBS program components, and 
make changes in educational attainment, job-related skills, employment and income, such 
changes could affect children favorably in these particular families. 

(4) -That there would be no effects on children . Changes may occur within the family, but 
not in aspects of family life that affect children immediately. For example, it may not be 
enough (from the point of view of impacts on children) to change mothers’ employment 
status or the sources of family income if these do not result in changes in the child’s 
immediate environments of home or child care. Perhaps changes in the home 
environment or child care do occur but are too small to result in impacts on child 
development. There could also be influences on aspects of family life going in opposing 
directions, with these influences resulting in no net change in children’s development. 

For example, maternal earnings might increase, but the home environment might 
simultaneously deteriorate with increased maternal stress. These different possibilities 
point to the importance of examining not only whether JOBS affected children, but also 
what kinds of changes occurred within families, and how these changes jointly help to 
shape impacts on children. 

The study was designed with sufficient power to reasonably assess whether there were 
harmful impacts on children, an important hypothesis to examine when the new provisions of the 
Family Support Act were being implemented. The design also has sufficient power to assess 
whether favorable impacts occurred or whether different impacts occurred for different 
subgroups. 

At what point should policymakers take findings on children into account in making policy 
decisions? 



In the present study, we report all program impacts on children that are statistically 
significant . These program impacts are reliable: they are very unlikely to have occurred just on 
the basis of chance. As such, these program impacts warrant continued monitoring. In the 
present study, we will want especially to monitor whether the kinds of outcomes on which 
statistically significant impacts were found at the two-year point continue to show differences at 
the final follow-up (five years after the families enrolled in the evaluation), and if such 
differences grow in magnitude. 

We also report on whether a statistically significant result meets a further criterion: that of 
"policy relevance." At the start of the study and as the study proceeded, researchers and policy 
makers met to grapple with the question of the point at which child impact findings should be 

SR-4 




18 



taken into account in considerations about policy. A decision was made that statistically 
significant findings that were of a particular magnitude should be considered relevant to policy 
discussions. Specifically, for this study, a policy relevant impact was defined as one in which the 
effect size was at least one-third of a standard deviation on a given measure. 

This threshold sets aside impact findings that are so small that, while they are reliable 
statistically and warrant continued monitoring over time, may at this point in time have limited 
importance in terms of children’s development. At the same time, the threshold for policy 
relevance does not require that an impact be "large" in magnitude 4 in order to meet the criterion. 
By setting the threshold in this way, we can be reasonably confident that we are being inclusive 
in identifying instances of possible harm as well as of possible beneficial effects on children, 
without focusing on effects that are so small as to be of limited importance for children s 
development. 

In presenting results, we discuss not only the significance and policy relevance of impacts 
but also the patterning of findings. We also identify those impacts for which effect sizes 
substantially exceeded the threshold for policy relevance, that is, were .50 or larger (which is 
considered "moderate" to "large" in magnitude; Cohen, 1988). The strongest evidence on which 
to base conclusions about impacts on children is a consistent patterning of impact results, 
particularly when impacts meet or exceed the criterion for policy relevance. A patterning of 
results, for example, might show consistently favorable impacts for families in a particular site, 
or a particular program approach. A patterning of results might also pertain to a type of child 
outcome, with findings in one aspect of development (such as health) consistently affected 
favorably (or unfavorably) across programs. 

What is the relevance oj_ t_heM. Gildings in the new policy context ? 

Data collection for the two-year follow-up in the present evaluation was completed in 
January of 1996, prior to implementation of the new welfare law. Are findings from this 
evaluation still relevant in the broader sense of having implications in the new policy context? 
With the passage of the new welfare law, the Personal Responsibility and Work Opportunity 
Reconciliation Act of 1996 (PRWORA), AFDC and the JOBS programs were supplanted by the 
Temporary Assistance to Needy Families (TANF) program. Nevertheless, the experiences of 
families participating in JOBS offers guidance to states implementing their own welfare-to-work 
programs. In particular, findings from the evaluation of JOBS may suggest which programmatic 
approaches affect child outcomes, and in what ways. Apart from program impact findings, the 
Child Outcomes Study also provides much information about the developmental status of control 
group children, and about the family contexts of these children; information indicating how these 
children and families would be faring in the absence of the JOBS Program. This information, 



4 Researchers in the behavioral sciences often rely on Cohen’s (1988) characterization of effect sizes (in 
standard deviation units) of ,20 as "small," .50 as "medium," and .80 as large. 



too, can help to guide efforts to shape programs that include the needs of both children and adults 
in families receiving public assistance (see Knitzer, Cauthen and Kisker, 1999). 

The present welfare policy environment differs substantially from that under AFDC in 
several important respects (see Zaslow, Tout, Smith and Moore, 1998). For example, welfare 
benefits are no longer considered an entitlement under the 1996 law, and states have substantial 
latitude in implementing their particular programs. Under TANF almost all states emphasize a 
quick transition to employment. The new federal law requires that recipients of public assistance 
be working within two years of the time they start to receive assistance (or sooner at state 
option), places a lifetime limit of 60 months on the receipt of federal TANF funds, and requires 
states to have a certain proportion of their caseloads meeting work requirements. But there are 
also greater supports for work, with almost all states now treating earnings and assets more 
generously, and significantly more federal funds are available for child care. Another difference 
is that under AFDC, JOBS also encompassed program approaches focusing on remediation of 
basic education skills to improve job prospects. Whereas the Family Support Act required 
mothers of children three and over to participate in work-related activities (with a state option to 
lower the age to 12 months), the new law lowered the age for all states to 12 months (or younger, 
if a state chooses). Finally, while under the Family Support Act, mothers were exempt from 
participation in welfare-to-work activities if they were needed in the home to care for a sick or 
incapacitated family member (including a child), there is no such explicit exemption under 
PRWORA. 

Over the years, public assistance has become more and more predicated on custodial 
parents’ involvement in work or mandatory welfare-to-work program activities, as policymakers 
have sought to encourage adult self-sufficiency while also meeting the goal of fostering poor 
children’s well-being. Under TANF and AFDC, recipients of public assistance were required to 
take steps toward economic self-sufficiency or face a reduction in benefits (sanctioning). In 
many ways, programs implemented under PRWORA may be seen as an intensification of both 
the obligation and the opportunity dimensions of those implemented earlier, with respect to this 
common priority. 

The present study is one of only very few that examines impacts on the development of 
children in the context of welfare-to-work programs. Findings on what happened to children in 
the context of JOBS programs will help build toward an understanding of how the new policy 
context is affecting children. We will need to be cautious about extrapolating from the specific 
findings of the present study to outcomes for children under new policies. Yet addressing the 
general questions of whether children were affected even in a program directed primarily at 
economic outcomes in adults, and how children were affected, will provide us with information 
that will continue to be of vital importance. In addition, documenting the effects of labor force 
attachment and human capital development approaches will yield important evidence regarding 
the implications for children of very different welfare-to-work strategies. 
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The results from this study are an important piece of the emerging picture on the impacts 
of welfare on children and families. It will be important to put the findings from this study in the 
context of findings from other studies, such as the Project on State Level Child Outcomes (Child 
Trends, 1999), which examines the impacts on children of five state approaches to experimenting 
with welfare policies through obtaining waivers from various AFDC provisions. 

III. Study Approach 

How is the development of, children being studied in the Child Outcomes Study o f the National 
Evaluation of Welfare-to-Work Strategies ? 

The present report focuses on a sample of about 3,000 families with young children 
whose development is being followed over a period of five years in the Child Outcomes Study of 
the National Evaluation of Welfare-to-Work Strategies. The Child Outcomes Study focuses in 
depth on children’s development and family life in a subset of the approximately 55,000 families 
participating in the full seven-site evaluation of economic impacts of JOBS programs as part of 
the National Evaluation of Welfare-to-Work Strategies. 

The children whose development is being tracked over time were all preschool-age when 
their mothers enrolled in the evaluation (enrollment occurred between September 1991 and 
January 1994). The families in the study come from three of the seven sites of the National 
Evaluation of Welfare-to-Work Strategies: Atlanta, Georgia; Grand Rapids, Michigan; and 
Riverside, California. These are study sites in which two different JOBS program approaches 
were tested: a labor force attachment approach, which encouraged welfare recipients to make a 
rapid transition to employment, and a human capital development approach, which encouraged 
recipients to engage first in education and training as a longer-term strategy to improve 
employment prospects. 

What are the characteristic s of the families at enrollment in the Chitij Outcomes Study?. 

Looking at the characteristics of the families in the sample when they entered the 
evaluation, there are numerous indications of disadvantage. For example, for the sample as a 
whole, a majority of mothers had never been married, though there was substantial variation 
across the sites (71 percent of mothers in Atlanta, 59 percent in Grand Rapids, and 43 percent in 
Riverside had never been married). In addition, in each of the sites, a majority of the mothers 
had received welfare for two years or more (about 75 percent in Atlanta; 72 percent in Grand 
Rapids; and 65 percent in Riverside). Between a third and half of the mothers in the three sites 
had limited literacy according to an assessment completed at baseline (about 48 percent in 
Atlanta, 33 percent in Grand Rapids, and 35 percent in Riverside). More than a third of the 
mothers in the sample reported a moderate to high number of depressive symptoms (about 36 
percent in Atlanta; 43 percent in Grand Rapids; and 35 percent in Riverside), a prevalence of 
symptoms substantially greater than in community samples. 
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Yet in some respects the characteristics of the families contradict commonly held 
assumptions about families receiving welfare. For example, while it is common to think of 
mothers receiving public assistance as having low levels of education, many of the mothers in 
this sample had completed high school or received a General Educational Development (GED) 
degree. Indeed, in two of the study sites (Atlanta and Grand Rapids), a majority of mothers had a 
high school diploma or GED. Many think of families receiving public assistance as having large 
numbers of children, yet most of the families in the sample had only one or two children. Some 
view welfare as an intergenerational pattern. Yet in the present sample, most of the mothers did 
not report receiving welfare when they were themselves children (between one-fifth in Riverside, 
and one-third in Grand Rapids). Finally, most of the mothers in the sample agreed with the 
statement that "It’s wrong to stay on welfare if you can get a job, even a job you don’t like," 
suggesting positive attitudes about employment. 

Even though all of the mothers in the sample had applied for or were receiving welfare at 
the start of the evaluation, there was substantial variation in terms of key background 
characteristics. This variation underscores the importance of asking whether children were 
affected differently in families of different backgrounds. For example, there were sizeable 
subgroups within this sample who had and had not completed high school or a GED; with higher 
and lower reading and math literacy scores; with few and many symptoms of depression. 

How were the data / 'or this report collected? 

This is the second report from the Child Outcomes Study. The first report described the 
family context and the children’s development in one study site at the start of the evaluation. 5 In 
the current report, for the first time, the possibility of program impacts on young children is 
being examined. The results come from a follow-up interview carried out with the families in all 
three sites approximately two years after enrollment in the evaluation, when the children were 
about five to seven years old. The interviews were conducted in the families’ homes. During the 
course of the interview, a direct assessment of the child’s cognitive development was carried out, 
and mothers completed further measures regarding the children’s development. The interview 
also collected information about aspects of family life that may have been affected by the JOBS 
programs, and thus, may help to explain program impacts on the children (for example, measures 
of family economic circumstances, maternal psychological well-being, the home environment, 
and the children’s child care participation). 



5 The report of the Descriptive Study ("How Well Are They Faring? AFDC Families with Preschool-Aged 
Children in Atlanta at the Outset of the JOBS Evaluation") asked whether, overall, this group of children was at risk 
for poor developmental outcomes, and which factors were most closely associated with the development and well- 
being of the children (Moore, Zaslow. Coiro, Miller, and Magenheim. 1995). This study identified risk factors for 
poor developmental outcomes in the children, but also protective factors associated with more positive 
development. 
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The measures we report on here from the two-year follow-up, particularly those 
pertaining to the children’s behavioral and emotional adjustment and health and safety, while 
often well validated in other research, must be seen as having the limitation of relying 
exclusively on mothers’ reports. For example, in the area of health and safety, we do not have 
doctors’ assessments of overall health, nor do we have hospital records data concerning 
emergency room visits. Mothers’ reports carry with them all of the influences that can affect 
maternal perceptions. For example, a mother who is concerned about being able to make it to 
work (particularly in a job without benefits such as sick leave) may find child health issues more 
salient and, thus, any indications of the child’s compromised health (e.g., the common cold) may 
lead these mothers to rate the focal child’s overall health less favorably than mothers without 
similar employment concerns. 

In the future, results will be presented from a final follow-up with the Child Outcomes 
Study families and children. This follow-up was conducted about five years after the families 
enrolled in the evaluation, when sufficient time had passed for longer-term changes in the 
families and children in response to JOBS to have manifested themselves. This report will 
include, in addition to information on assessments of the children’s cognitive development and 
maternal report measures of the children’s health and safety, and behavioral and emotional 
adjustment, data from teacher reports of the children’s progress and adjustment in school, and 
children’s reports of their own behavior and engagement in school. 6 The direct assessments of 
academic achievement are also more extensive at the five-year follow-up point. 

The larger, seven site evaluation of primarily economic impacts in the National 
Evaluation of Welfare-to-Work Strategies includes a more limited set of questions addressed to 
mothers about the well-being of aH of the children in the family. Impact results for these 
questions in the larger evaluation sample are included in the two-year follow-up report focusing 
on economic impacts in the full sample (Freedman, Friedlander, Hamilton, Rock, Mitchell, 
Nudelman, Schweder, Storto, 2000). 

In addition, a brief Synthesis Report (Hamilton, with Freedman and McGroder, 2000) is 
also being made available, drawing together and integrating the two-year follow-up results from 
the Child Outcomes Study, and the set of brief measures focusing on all children in the family in 
the larger evaluation. 



bindings will also be reported in the future from a special in-depth study of parenting behavior (the JOBS 
Observational Study; see Zaslow et al., 2000). This study, carried out within a subset of the Child Outcomes Study 
sample, involved direct observation of mother-child interaction soon after baseline and again four and a half years 
after baseline. Fine-grained observational measures of parenting behavior are used to ask whether mother-child 
interaction was affected by assignment to one of the JOBS programs (the Atlanta human capital development 
program). The JOBS Observational Study was funded by the Foundation for Child Development, the William T. 
Grant Foundation, the George Gund Foundation, and an anonymous funder, with additional funds for pretesting of 
middle childhood observational measures provided by the U.S. Department of Health and Human Services. 
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IV. JOBS Welfare-to-Work Programs 



What were the_ key components present in JOBS_ welfare-to-work programs? 

The programs implemented under JOBS had four key components: (A) mandated 
participation in education, training, and/or employment activities; (B) messages about the 
importance of such activities; (C) enhanced case management services to direct and monitor 
clients’ progress; and (D) services to facilitate employment. 

All families, whether they had been randomly assigned to one of the JOBS program 
groups or to a control group, received AFDC benefits. Families assigned to the program as well 
as control group members were also eligible for child care subsidies and for Medicaid while 
receiving AFDC and for a year following a transition off of AFDC and into employment 
("transitional benefits"). Control group members were eligible for child care assistance, similar to 
that offered to program group members, if they were participating in work preparation activities 
in which they had enrolled on their own. 

However, only the families assigned to one of the JOBS program groups were exposed to 
the four key program components: 

A. Mandated Participation 

Participation in approved activities to enhance economic self-sufficiency was mandatory. 
Failure to participate in approved activities could result in sanctioning (i.e., a reduction in 
welfare benefits). The activities emphasized, and the sequencing of activities, differed in the 
labor force attachment and human capital development programs. 

Clients in labor force attachment programs were first assigned to "job club," which 
emphasized learning how to find a job through classroom instruction and experiential activities 
such as phoning employers to set up interviews. After three to five weeks, clients who had not 
secured employment either continued their job search, were assigned to basic education, or 
received short-term training. 

Depending on their educational level and work history, clients in human capital 
development programs participated either in a basic education program, for example, a high 
school completion program, or a General Educational Development (GED) program, or in 
vocational training or college. If they did not find employment upon completion of their first 
activity, it was intended that they enroll in job club, individual job searches, on-the-job training 
or other work experiences, or go on to obtain additional education or vocational training. 
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B. Messages 



Messages provided to clients in the labor force attachment programs (at orientation and 
through case management) emphasized finding a job quickly. Messages provided to clients in 
the human capital development programs emphasized building skills in order to find "better 
jobs. In addition, while both control group and program group mothers were provided 
information about child care (which, as described below, in specific sites, included suggestions 
about the type of child care to use), program mothers may well have had greater exposure to 
these messages to the extent that enhanced case management put them in greater contact with 
their case managers. 

C. Enhanced Case Management Services 

Clients worked with a case manager to develop a plan for progressing toward economic 
self-sufficiency. Client and case manager then together monitored progress in light of this plan. 
And while both control group and program group mothers were entitled to child care services, 
program mothers — by virtue of receiving enhanced case management and, thus, greater 
opportunities for more frequent interactions with their case manager - may have a received more 
sustained assistance in securing child care. 

D. Services to Facilitate Employment 

The messages and sequence of activities in the labor force attachment and human capital 
development programs differed, but both emphasized finding employment and both included 
services, such as job club and job search, to facilitate a transition to employment. 

Did JOBS, programs differ, across sites? 

States had latitude in the implementation of their JOBS programs within the framework 
of these key components. Accordingly, the sites in this study varied in their orientation and in 
how they implemented their JOBS programs. These differences are documented in detail in a 
discussion of program practices and characteristics in the three sites (Hamilton, Brock, Farrell, 
Friedlander, and Harknett, 1997). What follows is a brief summary of key differences. 

a In. Atlanta , case managers had relatively small caseloads, and they tended to see their role 

as supportive, actively seeking out services for clients. Even though the labor force 
attachment and human capital development programs in Atlanta were clearly distinct, 
case managers in Atlanta more than the other sites tended to give messages to participants 
in both programs emphasizing the importance of skills and education (human capital 
development-oriented messages), perhaps due to their experience running human capital 
development-type programs. They also tended to be somewhat ambivalent about 
imposing financial sanctions. Nevertheless, sanction rates in Atlanta were fairly high. In 
Atlanta, families were free to choose among child care options (unlicenced provider in a 
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home setting, licensed provider in a home setting, or child care center), but were reimbursed only 
for licensed care, and were encouraged to use center-based care, on the grounds that such care 
was of better quality and more reliable. Assistance in securing and paying for child care was also 
promoted as a benefit of participating in JOBS. 

• Grand Rapids’ human capital development program was unique in its up-front, in-depth 
assessment of clients’ needs. However, caseloads were relatively high, making it difficult 
to give personal attention and encouragement in either program. A greater emphasis was 
placed on monitoring client participation and imposing financial sanctions for 
noncompliance. Clients in Grand Rapids were not encouraged to use one form of child 
care over another and case workers did not actively promote assistance with child care as 
a benefit of participating in JOBS. 

• In Riverside, the welfare agency had extensive experience and success in r unn ing labor 
force attachment-type programs. Thus, while the labor force attachment and human 
capital development programs in Riverside were distinctly different, in general, case 
managers tended to emphasize to clients in both programs the importance of a quick 
transition to the labor force. Despite moderate to large caseloads. Riverside case 
managers were held responsible for the educational and employment outcomes of their 
clients. Case managers closely monitored clients’ progress, and they encouraged clients 
to succeed in their assigned activities. In addition, while staff did not hesitate to enforce 
participation requirements, clients were afforded many opportunities to come into 
compliance before financial sanctions took effect. Clients in Riverside were encouraged 
to use informal child care arrangements, in part because such care involved lower 
reimbursement rates, but also because it was felt that many clients preferred to leave 
children with friends or family members. In the Riverside site, though not the other sites, 
the human capital development program was only available for recipients considered in 
need of basic education (with no high school diploma or GED, non-English speakers, or 
those with low scores on baseline literacy or math test scores). 7 



Individuals in need of basic education, and those not in need of basic education were then assigned to 
different random assignment processes in the Riverside site. Those who were considered to be in need of basic 
education were randomly assigned to any one of the three research groups. However, those considered not in need 
of basic education could be randomly assigned only to the labor force attachment or control groups (see Hamilton et 
al., 1997). As a result, when contrasts of research groups are carried out in the Riverside site, those in the human 
capital development group are compared to control group members who are likewise considered in need of basic 
education, whereas members of the labor force attachment group (who could be in need or not in need) are 
compared to all control group members. By contrast, Atlanta and Grand Rapids did not take mothers’ basic 
education needs into account when randomly assigning them to a program or control group; thus, mothers in these 
sites’ human capital development programs were not as disadvantaged (with respect to literacy and/or educational 
attainment) as the mothers assigned to Riverside’s human capital development program. Accordingly, in 
considering the patterning of findings for human capital development programs across all three sites, the reader 
should keep in mind, not only how the sites differ in terms of population and economic conditions (see Chapter 3), 
but also differences in the experimental designs. 
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In light of these program variations, we use the term " program " in reference to one of the 
six specific programs implemented in the three study sites, reserving the phrase "the JOBS 
Program" only when discussing the federally mandated welfare-to-work program authorized in 
the Family Support Act. We use the term " program approach " to refer to the human capital 
development and the labor force attachment welfare-to-work strategies. In order to take the local 
variation in programming into account, we report on the impacts of each program approach 
(labor force attachment and human capital development) for each of the research sites separately. 
Comparing and contrasting the results from each of the three sites gives us a picture of the range 
of impacts on children of their mothers’ assignment to human capital development and labor 
force attachment program approaches as implemented in different locations, and with somewhat 
different emphases. It also directs us to consider different participant characteristics in each site 
and differences in the local job markets when interpreting program impacts. 

What was the extent oj_ mothers ’ exposure t Q. JOBS, programs ? 

Assignment to a labor force attachment or human capital development program under 
JOBS did significantly increase participation in appropriate activities for mothers in the Child 
Outcomes Study sample. Thus, mandatory participation requirements (combined with the JOBS 
programs’ messages, case management, and services), affected participation even among mothers 
with preschool-age children. 

Yet the duration of program exposure was generally not long. This is actually in keeping 
with program goals. If participation in such activities as job search or basic education had 
occurred for all or even most of the months of the two-year follow-up period, this would indicate 
that in this time frame, mothers were not progressing off welfare and to employmejft - the actual 
goals of the programs. In the Atlanta, Grand Rapids, and Riverside sites, in the full evaluation 
sample, the average duration of participation (among those who participated for at least a day 
following baseline), ranged from about 3 to 6 months of the two-year follow-up period in the 
labor force attachment programs, and about 6 to 9 months in the human capital development 
programs (Hamilton et al., 1997). 

In considering the occurrence and magnitude of program impacts of JOBS on children, it 
is important to keep in mind that exposure to JOBS programs, first, was generally much shorter 
than the periods of exposure often studied in evaluations of early childhood care and education 
programs (see Barnett, 1995); and, second, for the most part involved the direct exposure to the 
programs of mothers, not children. There were no program components like early childhood 
intervention addressed directly to the children. Thus, any impacts on children should be seen as 
transmitted through changes in family life brought about by mothers assignment and exposure to 
JOBS programs. These would include increases in the use of child care arrangements. 

Figure SR-1 reflects this understanding of how JOBS, a set of programs directed to 
mothers, could have affected children. The conceptual framework that is illustrated notes how: 
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• assignment to a JOBS program (A), could affect: 

• mothers’ exposure to. and experiences in. the JOBS program as the program was actually 
implemented (B): 

• and, through program exposure, could affect (C): outcomes directly targeted bv the JOBS 
program (such as maternal educational attainment, employment, and economic 
resources), and outcomes derivative of these targeted outcomes (such as use of child care 
and access to health insurance); 

• as well as (D): further outcomes not directly targeted bv JOBS programs (such as 
maternal psychological well-being, and parenting); 

• and through changes in these aspects of family life and children’s child care situations, 
affect child outcomes (E). 



Figure SR-1: Conceptual Framework 



Targeted Outcomes 




Non-Targeted Outcomes 



We turn now to a summary of the findings. The findings are presented in three sections. 
Section V summarizes results for the children in the sample: first, how children in the control 
groups were doing in the absence of JOBS program influences, and second, whether the JOBS 
programs altered this course of development. Section VI follows the same sequence regarding 
families, first asking how control group families were progressing in the absence of program 
influences, and then whether the JOBS programs had affected family life. Section Vll reports 
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findings regarding the linkages between program impacts on families and children, aslorng w c 
aspects of family life help to explain selected findings regarding program impacts on children. A 
final section, Section VIII. summarizes the findings and discuss their implications. 

V. Findings for Children: 

iWplnpmgnt and Well-Being of Co ntrol Group Children, 
and Program Impacts on Children 

We first describe findings at the two-year point for children in the control groups of the 
Child Outcomes Study, as a background for understanding child impact results that follow. 

A. Children in the Control Groups 

How were children in the control grotm of the t hree st u dy sites taring at the tune o£the tw oz 
year follow-up? 

In the absence of JOBS program influences, the children in the sample were at risk of 
poor developmental outcomes, especially in the area of cognitive development. 

For example, as can be seen in Figure SR-2, children in the control groups were 
significantly less cognitively ready for school than a national sample of children of the same age, 
as measured by standardized scores on an assessment of cognitive school readiness, the School 
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Figure SR-2 

Mean Standardized Score on Bracken School 
Readiness Composite (0-19) 




National 
Standardization 
Sample 



Child Outcomes Study 
Control Groups 
(focal child) 



Sources: Bracken. B. (1994). Bracken Basic Concept Scale - Examiner's Manual (Appendix B). The Psycnotoflical Corporation. 
Chad Outcomes Study calculations by Chad Trends. 
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Readiness Composite of the Bracken Basic Concept Scale (Bracken, 1984). Children in the 
control groups also had higher average scores on a measure of behavior problems than age-mates 
in a national sample (Figure SR-3). On the Behavior Problems Index (Peterson and Zill, 1986; 
Zill, 1985), they tended to have more frequent behavior problems overall, and more frequent 
externalizing (aggressive/acting out) behavior problems in particular, than five- to seven-year- 
olds in the National Longitudinal Survey of Youth-Child Supplement. Differences here are not 
as marked, however, as the differences in cognitive school readiness, though they are statistically 
significant. 



Figure SR-3 

Mean Frequency of Total Behavior Problems (0-2) 
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Sources: Calculations by Child Trends. 



At the same time, as can be seen in Figure SR-4, mothers rated the overall health of focal 
children in the control groups as favorably, or more favorably, than mothers of children aged 
five- to-seventeen in a national health survey. 8 However, as we have noted, the Family Support 
Act exempted mothers who were needed in the home to care for a sick or incapacitated family 
member, including a child, from mandated participation in JOBS. Thus, it is not surprising that 
mothers of children in the control groups of the Child Outcomes Study sample rated their 
children’s overall health in this way. 



8 Mothers were asked to rate the focal child’s overall health in the following question: "Would you say that 
(CHILD’S) health in general is: excellent, very good, good, fair, or poor?" 
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Figure SR-4 

Percent of Mothers Rating the Focal Child's General Health 
as Very Good or Excellent 
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Sources: Current estimates from the Notional Health Interview Survey, 1993. Vital and Health Statistics (Series 10, No. 190). Teble 70. 
Child Outcomes Study calculations by Child Trends. 



B. Program Impacts on Children 

Were there program impacts on measures of children ’s development and wellbeing, at the 
time of the two-vear follow-up? 

Did mothers’ assignment to a JOBS program exacerbate this developmental risk in 
children? Alternately, did it diminish risk? Was it the case that a program directed primarily to 
mothers had little or no impact on children? Or were there impacts that varied in direction 
(favorable and unfavorable) according to the type of child outcome, the research site, program 
approach, or the characteristics of the families? We summarize impact findings first "overall" or 
in the "aggregate." That is, within each study site, we ask whether the measures of child well- 
being and development differed for the human capital development group as a whole as opposed 
to the control group as a whole, and for the labor force attachment group as a whole as opposed 
to the control group as a whole. In subsequent sections we turn to the question of whether 
children in families who entered the evaluation at higher and lower risk showed different patterns 
of program impacts. Impact analyses control for key background characteristics, including 
family race/ethnicity and child gender. Detailed reporting on child impacts by race/ethnicity and 
child gender are planned in the future. 

In analyses at the aggregate level there were relatively few statistically significant 
program impacts on children. Further, those impacts that did occur tended to be small in 
magnitude. In fact, only one of the impacts reached the criterion of policy relevance. The 
findings indicate that the welfare-to-work strategies implemented under JOBS did indeed have 
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impacts on selected child outcomes, though it is important to underscore that these aggregate 
impacts were relatively few and small in magnitude. 

What were the program impacts in differenj aspects of children ’s development^ 

Table SR-1 provides a brief summary of the number and direction ("+" indicating 
favorable, and indicating unfavorable) of statistically significant program impacts for each of 
the aspects of children’s development studied (cognitive development and academic 
achievement, behavioral and emotional adjustment, and health and safety) and for each of the six 
programs studied (the human capital development and labor force attachment program 
approaches in each of the three sites). This table provides an overview, summarizing impact 
findings across discrete measures for each aspect of development. Some of the specific measures 
summarized in this table pertain only to the focal child (for example, the assessment - of the focal 
child’s cognitive school readiness, the measure of behavior problems, and the rating of overall 
health described above). However, some of the specific measures summarized in the table 
pertain to all of the children in the family. For example, mothers were asked whether any of the 
children in the family had been suspended or expelled from school, and whether any of the 
children in the family had had an accident, injury, or poisoning requiring emergency medical 
attention. Thus, the table includes both "focal child" measures and "any child" measures. 9 

The table notes the number of possible impacts (that is, the total number of measures of 
each aspect of development considered, across all six programs), as well as the number of 
statistically significant impacts actually found . The contrast of actual and potential number of 
impacts underscores our conclusion that there were relatively few impacts overall, though more 
than one would expect to occur on the basis of chance. Overall, there were 1 8 statistically 
significant treatment-control group differences in measures of child development and behavior, 
out of 132 impact analyses conducted across all child outcomes and programs. While the 
number of significant impacts is relatively small, nevertheless, it is a greater number of findings 
than we would expect by chance alone 10 , indicating that JOBS welfare-to-work programs did 



9 Chapter 2 of the report provides details about the specific "focal child" and "any child" measures. Chapter 
6 presents impact results for specific child outcome measures. 

l0 Some would argue that each statistically significant finding is valid irrespective of the number of 
analyses conducted. Others would argue that the greater the number of analyses conducted, the greater the 
likelihood of chance findings and. thus, one needs to take the number of analyses into account. In order to minimize 
the likelihood of reporting chance findings, we chose to calculate the proportion of statistically significant impacts 
across all child outcome measures and across all six programs. Given that the experiment-wise Type I error rate 
was set at .10, we might expect to find significant results 10 percent of the time due to chance alone, translating into 
just over 13 statistically significant impacts across the 22 child outcome measures and the six programs (22 x 6 = 
132, 132 x .10 = 13.2). Some argue that a more stringent standard is needed, requiring that the number of 
significant impacts within each program must exceed chance levels, or that the number of significant impacts within 
each domain of child development must exceed chance levels. Because there is a lack of consensus on this issue 
among statisticians, and given that a goal of the Child Outcomes Study was to provide a thorough examination of 
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indeed have impacts on selected child outcomes. 



As can be seen, child impact findings at the aggregate level differed in direction 
according to aspect ("domain") of development. All of the impacts in the domain of children’s 
cognitive development and academic achievement wer e favorable. That is, when there were 
statistically significant differences, it was always the case that children whose mothers had been 
assigned to a JOBS program scored better on measures of this aspect of development than 
children whose mothers had been assigned to a control group. Moreover, at least one favorable 
program impact on a measure of cognitive development and academic achievement occurred in 
four of the six programs examined here. 

By contrast, all of the statistically significant impacts in the area of child health and 
safety were unfavorable. That is, when there were statistically significant impacts, they always 
occurred in a direction indicating that children of mothers assigned to a JOBS program had less 
favorable scores than children of mothers assigned to a control group. 

All of the unfavorable health impacts occurred in the Riverside site, and unfavorable 
health impacts occurred in this site in both programs (labor force attachment and human capital 
development). Mean scores indicated that program group mothers in the Riverside site rated the 
focal children as less healthy overall. In addition, a smaller proportion of mothers in the program 
groups than in the control groups rated the focal children as in excellent or very good health (as 
opposed to good, fair or poor) on this overall health rating. The unfavorable impact of 
Riverside’s labor force attachment program on the proportion of focal children rated as in very 
good or excellent health was large enough to be considered "policy relevant." 

These unfavorable impacts indicate that a larger proportion of program than control group 
mothers in the two Riverside programs rated focal children in the range of good, fair, or poor 
health (rather than very good or excellent). We examined the origin of this impact more closely, 
asking especially whether the higher proportion of experimental group mothers rating their 
children in this range was largely attributable to the "good" rating, and thus did not indicate a 
shift toward fair or poor health. However, the findings in fact reflect a higher proportion of 
mothers rating their children as in poor or fair health in the two programs. 

No statistically significant differences were found at the aggregate level in any of the 
research sites on the proportion of mothers reporting that an accident or injury had occurred to 
any of their children that was severe enough to require emergency medical attention. Thus, 
differences in the area of health and safety occurred in terms of mothers’ ratings of focal 



program impacts, we did not adhere to this more stringent standard. Moreover, our examination of impacts at the 
subgroup level (see Chapter 7) and of pathways through which particular impacts on children appear to have come 
about (see ChapterlO) provides further evidence that even rare impacts are not necessarily chance findings. 
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children’s overall health, but not on outcomes related to emergencies for any of the children in 
the family. 

The overall health rating reflects the mothers’ perceptions of the focal child’s health. We 
do not have doctors’ assessments of the child’s overall health or the incidence and/or severity of 
concrete illnesses, such as ear infections. Maternal perceptions may, in part, reflect the salience 
of this issue. For example, employed mothers may be concerned about the possibility of missing 
work because of a child’s illness, and, thus, any indications of the child’s compromised health 
(e.g., the common cold) may lead employed mothers more so than non-employed mothers to rate 
the focal child’s overall health less favorably. At the same time, lower health ratings could, in 
fact, reflect poorer child health, which could result, for example, from exposure in child care 
settings to other children with health symptoms, from diminished attention to such family 
routines as bed time, or from mothers’ having less time to bring children to health care providers 
for diagnosis and treatment. In either case, it is possible that lower health ratings by program 
mothers are, to some extent, derivative of maternal employment. (See Section VII below for 
results suggesting the degree to which maternal employment and/or child care did, in fact, help to 
explain unfavorable impacts on this rating of focal children’s health.) 

Impacts on measures of the children’s behavioral development and emotional adjustment, 
in contrast with measures of cognitive development and health, included both favorable and 
unfavorable impacts. Unfavorable behavioral impacts occurred in each site’s labor force 
attachment program, whereas there does not appear to be a pattern (by site and/or program 
approach) to the occurrence of favorable behavioral impacts. 

Should_ we have expected- child_ impact findings, to go in the_ same direction across all aspects of 
development? 

Is it problematic that findings went in different directions for different aspects of 
development, with impacts on cognitive development and academic achievement favorable; 
health and safety unfavorable; and behavioral and emotional adjustment including both favorable 
and unfavorable impacts? In fact, there are precedents in other bodies of research on children’s 
development for such complex patterns. Child care research provides an example. Findings in 
this literature show that children who have participated in formal child care settings tend to have 
better scores on measures of cognitive development (e.g., Zaslow, Oldham, Magenheim and 
Moore, 1998), mixed outcomes on measures of behavioral development (e.g., Barnett, 1995; 
Yoshikawa, 1995), and some unfavorable outcomes in terms of health (such as more ear and 
intestinal infections, e.g., Hirsh-Pasek, 1998). In giving this example, our intent is not to suggest 
that impacts of JOBS programs on children are transmitted primarily through child care 
influences. Rather, our intent is to point out that different aspects of development can be affected 
in differing directions by a developmental context (like child care, or mothers’ assignment to a 
JOBS program). 
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What patterns were observed, with respect to the three sites and_ six pros rams? 



Child impact findings at the aggregate level tended to differ according to site, as shown in 
Table SR- 1. 

Cognitive development: Favorable impacts were found in Atlanta’s labor force 
attachment program and in all three human capital development programs . Favorable 
program impacts on children’s cognitive development and academic achievement occurred 
especially in Atlanta’s labor force attachment program. In this program, children had 
significantly higher mean cognitive school readiness scores, as well as a more favorable 
distribution of scbres on the assessment of cognitive school readiness (i.e., the proportion of 
children scoring at the high end of the distribution was greater, and the proportion of children 
scoring at the low end was smaller). 

In addition, at least one favorable impact in the domain of cognitive development 
occurred in each site’s human capital development program. Specifically, in Atlanta and Grand 
Rapids, children in the human capital development program had scores indicating a more 
favorable distribution on the Bracken assessment (i.e., the proportion of children scoring at the 
high end of the distribution was greater). In the Atlanta and Riverside sites, a smaller proportion 
of mothers in the human capital development program than in the control group reported that any 
of the children in the family had had an academic problem since the start of the evaluation. 

It is possible that programs that improve mothers’ educational outcomes provide the 
mothers themselves with a source of cognitive stimulation. This in turn may influence how 
much and how mothers interact with their children, with implications for the children’s cognitive 
development. Children may also be influenced by observing their mothers pursuing educational 
goals. Mothers who themselves participate in educational activities may place more emphasis on 
school work, and/or may feel more effective in assisting their children with homework. 

The concentration of favorable cognitive impacts on children in each site’s human capital 
development program and in Atlanta’s labor force attachment program is consistent with this 
hypothesis. These four programs increased mothers’ educational attainment (e.g., the receipt of a 
high school diploma or GED, the receipt of a trade degree). 

Health: Unfavorable impacts were found only in Riverside . All of the unfavorable 
child health impacts at the aggregate level occurred in the Riverside site. Unfavorable impacts 
occurred in both programs in this site, and all findings pertained to the focal child’s health, as 
rated by the mother. One of these impacts was large enough to be considered policy relevant. No 
unfavorable health impacts were statistically significant at the aggregate level for either of the 
programs in the other two study sites. As noted above, no program impacts were found at the 
aggregate level on the more extreme measure of accidents, injuries or poisonings requiring a visit 
to an emergency room or clinic. Further, as noted above, the maternal report measure of overall 
health likely reflects not only the actual occurrence of particular health symptoms in the focal 
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child, but also the salience of the focal child’s general health status (which may vary as a 
function of maternal employment status and history). 

Behavior: Unfavorable impacts were found only in labor force attachment 
programs, though favorable impacts were also found in Atlanta’s labor force attachment 
program. Each of the labor force attachment programs studied had at least one unfavorable 
program impact in the area of behavior and emotional adjustment as reported by the mother. 
However, the picture was mixed for the labor force attachment program in Atlanta. For this 
program there were also two favorable impacts on measures of behavior and emotional 
adjustment. (A single favorable. behavioral impact also occurred in Riverside’s human capital 
development program.) 

Pattern of findings in Atlanta and Riverside . On balance, impacts in Atlanta were 
favorable for each program approach. Of eight statistically significant child impact findings in 
Atlanta, seven were favorable (with the single unfavorable impact in this site occurring on a 
behavioral outcome for children whose mothers were in the labor force attachment program). 

By contrast, on balance, in the Riverside site, significant child impact findings were 
unfavorable. Of seven statistically significant child impacts in the Riverside site, five were 
unfavorable. The unfavorable findings in Riverside were concentrated in the health area, but one 
also occurred on a behavioral outcome for children whose mothers were in the labor force 
attachment program. 

We started. QML with OR interest in exploring possible favorable^ as_ well 31 unfavorable program 
impacts on children. What can we conclude about fayorab/e impacts? 

In the context of previous research on welfare-to-work programs, it is important to note 
that the aggregate impact findings in the present study include evidence of favorable impacts on 
children, particularly in the cognitive domain of development, and particularly in Atlanta. 

Findings at the aggregate level indicate slightly more favorable than unfavorable 
significant impacts on child outcome measures. All of these favorable impacts may be 
considered small in magnitude, with none reaching the threshold for policy relevance. They are 
concentrated in the domain of cognitive development. Each site’s human capital development 
program had at least one favorable impact in the cognitive domain, but the greatest concentration 
of favorable cognitive impacts occurred in Atlanta’s labor force attachment program. 

To date, only three evaluations of welfare-to-work programs with rigorous experimental 
designs have had an explicit focus on child impacts: the present study, the Teenage Parent 
Demonstration (Kisker, Rangarajan and Boiler, 1998), and the New Chance Evaluation (Quint, 
Bos and Polit, 1997). The Child Outcomes Study is the first noting a pattern of favorable child 
' impacts at the aggregate level for cognitive outcomes, and a greater number of favorable than 
unfavorable impacts, on balance, at the aggregate level. 
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Favorable impacts on children, and especially boys, were recently reported in a program 
evaluation of a different kind, however: the New Hope Evaluation (Bos, Huston, Granger, 
Duncan, Brock, and McLoyd, 1999). New Hope is a demonstration program in which low- 
income adults, already working or willing to work 30 or more hours a week, are assured of an 
income above the poverty level, along with benefits such as health insurance and child care 
subsidies. Participants in this program, in contrast with those in the three welfare-to-work 
demonstrations described above, were not all receiving welfare, were not all women, and they 
did not all have children. Also, unlike current and recent welfare-to-work programs, 
participation in New Hope was voluntary. Children, and especially boys, were more likely to 
have been in organized after-school activities and were reported by their teachers to be doing 
better academically and to be showing more positive social behavior. Parents in the program 
group reported more positive social behavior in their sons, and the boys themselves had higher 
aspirations and expectations in terms of future occupations and advanced education. 

Perhaps New Hope and the JOBS programs examined here, to a greater extent than New 
Chance or the Teenage Parent Demonstration, affected family economic activities and resources, 
child care use, or the stimulation and support available in the home environment. Such impacts 
on families might help to explain the presence of favorable child impact results. We will return to 
these possibilities for the Child Outcomes Study in examining the evidence on the processes 
through which the child impacts arose (Section VII). 

How were children j rom higher - qnd_ lower-risk families affected? 

As noted above, the Child Outcomes Study was initiated in order to examine four 
diverging possibilities: that there would be favorable program impacts on children, unfavorable 
program impacts on children, no net impacts overall, or impacts especially for particular 
subgroups. We turn now to the fourth possibility, examining child impacts occurring for families 
with particular characteristics. 

Previous research examines whether welfare-to-work programs and other kinds of 
interventions have different effects on children according to whether or not families have 
particular risk factors and also according to the total number of risk factors they have. For 
example, the New Chance Demonstration (Quint et al., 1997) found unfavorable impacts on 
measures of child behavior problems as perceived by the mother. Unfavorable impacts occurred 
especially for families in which the mother had entered the evaluation at high risk of depression, 
in which the mother had been out of school two years or more, and in families with more risk 
factors overall. The Infant Health and Development Program, which aimed at improving the 
cognitive development of children born at low- weight, provided center-based early intervention, 
home visits, and parent support services. This program had positive effects for children from 
poor families when these families had no or few risk factors, but not when poor families had 
multiple risk factors (Brooks-Gunn, 1997). Thus there is evidence that children in low income 
families respond differently to interventions according to the presence of particular risk factors, 
and the total number of risk factors, in their families. 
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Following in this tradition, in the present study we looked at child impacts in light of the 
presence of particular risk factors in families and also the total number of risk factors in families. 
We defined risk in terms of characteristics found in previous research on welfare families to be 
associated both with greater difficulty for the mother in making a transition to employment and 
also with a less positive course of development in children, apart from any intervention. 

Research on factors that impede employment among families receiving welfare ("barriers 
to employment"), point to the importance of demographic characteristics of the family (such as 
having more children in the family), human capital factors (such as limited educational 
attainment and low basic skills; lack of work experience and history of longer welfare receipt), 
and psychosocial factors (such as maternal depressive symptoms) (evidence reviewed in Zaslow, 
Hair, Dion, Ahluwalia, and Sargent, 1999). These same factors have been found to be associated 
with the well-being and development of young children in families receiving welfare (Moore et 
al., 1995). Thus, there is evidence that family configuration, educational background and skills, 
work and welfare history, and maternal psychological well-being, are associated both with 
employment and with children’s development in samples of families receiving welfare, apart 
from welfare-to-work programs. 

It is not clear from the existing research, however, which type or types of risk may be 
important to the pattern of program impacts for children in light of mothers’ assignment to a 
JOBS welfare-to-work program. Accordingly, we examined program impacts on children in 
light of a number of different types of risk. Each family within the study was categorized as at 
higher or lower risk in four different ways: 

(1) sibling constellation risk (with higher risk involving more children and/or more 
closely spaced children); 

(2) educational risk (with higher risk involving limited maternal educational attainment or 
low maternal math or reading literacy scores); 

(3) work risk (with higher risk involving limited maternal employment history, multiple 
barriers to employment, or a history of welfare receipt of five or more years); and 

(4) maternal psychological well-being risk (with higher risk involving mothers with more 
indications of psychological distress). 

Families could be categorized as at higher risk on more than one category. For example, a 
family might fall into the higher-risk category for all four types of risk, or might be categorized 
as at higher risk in terms of sibling constellation and maternal psychological well-being risk (but 
not in terms of educational and work risk). By studying each of these types of risk, we could ask 
whether program impacts on children were especially linked with risk on one (or more) 
dimension. 
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In keeping with the previous research, we also created a summary variable looking at the 
number of these risk factors for which each family was in the higher-risk category: 

(5) cumulative risk (with higher risk involving a larger cumulative number of the 

preceding four categories for which the family fell into the higher-risk category). 

This made it possible to ask whether program impacts on children differed according to whether 
the family had a greater or lesser total number of risk factors. 

Finally, we also explored a measure of risk that we thought might be informative, but for 
which there was little precedent in the research on families receiving welfare. The research on 
maternal employment documents differences in outcomes for children according to whether the 
mother’s work role is concordant or discrepant with her preferred role (Zaslow, Rabinovich, and 
Suwalsky, 1991). Extending this to the present study, we asked whether impacts of JOBS on 
children would differ according to whether, at baseline, mothers felt more or less positively about 
working. On a more exploratory basis, then, we examined reservations about working (with 
higher risk involving mothers who tended to endorse statements indicating a preference for 
taking care of family over finding employment). 

The question that the subgroup impact analyses address is that of whether risk factors that 
are associated with less progress toward employment in mothers, and less positive development 
in children (in the absence of any programs or interventions), are also associated with different 
patterns of impacts for children in the context of JOBS welfare-to-work programs. Given the 
priority in the Child Outcomes Study of providing a rigorous examination of the possibility of 
unfavorable impacts for children, careful scrutiny must be given to the possibility that 
unfavorable effects occurred for children in certain subgroups. An important possibility is that 
unfavorable impacts might occur especially for families at higher risk, both in terms of specific 
risk factors and/or on the overall number of risk factors. 

While this prediction must be examined carefully, it is not the only pattern that can be 
anticipated. Indeed, JOBS programs were intended to address the difficulties faced by families 
in making a transition to employment, through a combination of supports and requirements. The 
possibility exists that especially for families with risk factors, one or both JOBS programs might 
be especially helpful, with children benefitting accordingly. 

The findings for subgroups did not indicate any one approach to defining risk as more 
important than the others. Indeed, the pattern of findings generally held across each of the 
different approaches to defining risk. 11 Accordingly, we discuss below the pattern of findings for 



1 'The full report discusses the more subtle ways in which subgroup impact findings differed across the 
approaches to defining risk (sibling constellation risk, educational risk, work risk, maternal psychological well- 
being risk, and cumulative risk; with an exploratory examination of reservations about working as well). 
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higher-risk families overall, and for lower-risk families overall. 

As for the aggregate impact findings, the impact findings for subgroups were not 
pervasive; that is, there were relatively few statistically significant program impacts for particular 
higher- or lower-risk subgroups. Yet when looking separately at findings for each particular type 
of risk (sibling constellation risk, educational risk, work risk, maternal psychological well-being 
risk, cumulative risk, or reservations about working), the number of statistically significant 
differences exceeded what would be expected to occur simply on the basis of chance. 12 We 
begin with the question of whether there was a pattern of unfavorable impacts for children in 
higher-risk subgroups. However, we also explore the possibility of a pattern of favorable impacts 
for children in higher-risk subgroups, and we summarize findings for lower risk subgroups as 
well. 



Higher-risk subgroups . The findings for higher-risk subgroups do not support the 
simple and straightforward hypothesis that there would be a concentration of unfavorable 
impacts for children in higher-risk families. Rather, the findings are in accord with a more 
complex pattern. We underscore that as for the aggregate impacts, statistically significant 
findings were relatively few. 

Findings for children in higher-risk families were generally favorable when the family 
had been assigned to a human capital development group or to Atlanta’s labor force attachment 
program. The only exceptions relate to unfavorable health impacts of Riverside’s human capital 
development program (though all other findings for higher-risk families in this program were 
favorable), a single unfavorable health impact of Atlanta’s labor force attachment program, and a 
single unfavorable behavioral impact of each of Atlanta’s JOBS programs. By contrast, impacts 
for children in higher-risk families assigned to either Grand Rapids’ or Riverside’s labor force 
attachment program tended to be unfavorable. 

Impacts for children in higher-risk subgroups tended to be small in magnitude. Yet 
several of the findings, all of these unfavorable and occurring in labor force attachment 
programs, were of sufficient magnitude to be called policy relevant. 

That largely favorable impacts were found for children from higher-risk families when 
mothers had been assigned to human capital development programs or to Atlanta’s labor force 
attachment program may be of particular interest to policy makers. In the new policy context, as 
caseloads drop, there are concerns that perhaps families remaining on welfare, like the higher- 
risk families studied here, face barriers to making the transition from welfare-to-work. Programs 



l2 The proportion of findings to reach statistical significance relative to the number of impacts examined, 
exceeded what might be expected on the basis of chance for all six approaches to defining risk. The general pattern 
of findings that we discuss for higher- and lower-risk subgroups held across the different approaches to defining 
risk. 
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that have favorable impacts on mothers’ education or training may help address barriers, and in 
doing so, also benefit children. 

Lower-risk subgroup s. For children in lower-risk subgroups, there was a concentration 
of unfavorable impacts in three programs: both of Riverside’s programs, and the labor force 
attachment program in Grand Rapids. Across these three programs, a number of the statistically 
significant impacts were large enough to meet the criterion for policy relevance. In all but one 
instance, these policy relevant impacts were unfavorable. Further, many (one-third) of the 
unfavorable and policy relevant impacts found for children in lower-risk families substantially 
exceeded the threshold for policy relevance, in that effect sizes were .50 or larger (which is 
considered "moderate" to "large" in magnitude; Cohen, 1988). 

On balance, the significant impacts for lower-risk families in Atlanta’s labor force 
attachment program were favorable. None of these impacts was of a magnitude to be called 
policy relevant. Lower-risk families in human capital development programs outside of 
Riverside showed either no impacts or only isolated impacts, mixed as to direction. 

In all three programs in which there were unfavorable and policy relevant impacts for 
lower-risk families, such impacts occurred both in the domain of child health and also in the 
domain of child behavior. In these analyses, there is also evidence of unfavorable health impacts 
of sufficient magnitude to be called policy relevant occurring beyond Riverside. In general, the 
subgroup impact analyses, in contrast with the aggregate impact analyses, provide evidence of 
more widespread health impacts. Significant, although not always policy relevant, impacts in an 
unfavorable direction went beyond Riverside in these analyses. Indeed they occurred for lower- 
risk subgroups in all programs and sites with the exception of Grand Rapids’ human capital 
development program. Ail health impacts for children in lower-risk families were unfavorable. 

In the Riverside site, children in lower-risk families assigned to the labor force 
attachment program also showed unfavorable (and in some instances policy relevant) program 
impacts in the cognitive domain. This was the only set of unfavorable impacts in the cognitive 
domain across all of the subgroup impact (and aggregate impact) analyses. 

In sum, the findings for lower-risk subgroups are not pervasive (that is, the proportion of 
tested impacts to reach statistical significance was not high) nor do they appear specifically in 
one program approach or another (human capital development or labor force attachment). Rather, 
the larger unfavorable impacts are concentrated in three particular programs (Grand Rapids’ 
labor force attachment program, and both of Riverside’s programs). 

What are the implications of the child impact findings /or the concern about possible 
unfavorable impacts of JOBS welfare-to- work Qrograms on children? 

On the one hand, the dearth of policy relevant impacts at the aggregate level diminishes 
concern about unfavorable effects for children of mothers’ assignment to a JOBS welfare-to- 
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work program. Yet the concentration of unfavorable and policy relevant impacts, occurring 
especially among lower-risk subgroups in three programs suggests a need for continued 
vigilance. 

It was not the case that the subgroup impact findings were all large in magnitude. Yet 
some subgroup impacts were moderate to large in magnitude (i.e., were .50 or larger). Moreover, 
all of these moderate to large impacts were unfavorable, and all occurred for lower-risk 
subgroups. The fact that the unfavorable policy relevant impacts occurred in a concentrated 
manner (for lower-risk families in three specific programs), and that some of these were 
moderate to large in magnitude, suffices to indicate that we cannot entirely dismiss the 
possibility of unfavorable effects on children. 

We need to continue to follow the development of children in particular lower-risk 
subgroups over time. In analyses of the five-year follow-up data in the future, it will be critical 
for understanding the impacts on children of these programs to ask whether the impacts for these 
subgroups are sustained, grow in magnitude, diminish, or disappear and to examine the context 
of such findings. For example, if unfavorable impacts are sustained, it will be important to 
explain whether the impacts are occurring in the context of ongoing economic struggle for 
families trying to sustain employment at low wages. If, on the other hand, the unfavorable 
impacts for these subgroups disappear over time, it will be important to ask if an initial period of 
economic struggle gave way to one of increased and stable economic well-being. 

The findings suggest the need for continuing vigilance regarding the possibility of 
unfavorable impacts on children, though especially in particular programs and at particular risk 
levels. At the same time, the occurrence of favorable program impacts, albeit rarely large 
enough to call policy relevant, is encouraging and important to monitor over time. Favorable 
impacts were present especially for cognitive outcomes, and in Atlanta’s programs. In addition, 
there was a tendency for favorable impacts to occur for children from higher-risk families whose 
mothers had been assigned to human capital development programs or to the Atlanta labor force 
attachment program. 

VI. Findings: 

Outcomes for Control Group Families, 
and Program Impacts on Family Outcomes 

What was the family context for these child impact findings? We turn now to an 
examination of outcomes for families. As we did for the child outcomes, we first consider how 
families were faring in the control groups, in the absence of JOBS programs, and then turn to an 
examination of program impacts. 



SR-28 



O 

ERIC 



42 



A. Control Group Families 

How were families fating economically in the absence of_ a JOBS, program? 

Analyses focusing on control group families indicate that, in the absence of welfare-to- 
work strategies under JOBS, sample families would have made some educational and economic 
progress over the two-year follow-up period, though many would still be considered 
disadvantaged. 

Following random assignment, between 8 and 19 percent of control group mothers in the 
three sites received some sort of educational credential. Economically, whereas all sample 
families had applied for or were receiving AFDC at baseline, at the time of the two-year follow- 
up study, only 53 to 79 percent (with the percent varying by site) of control group mothers were 
receiving welfare. More mothers in the control groups (between 29 and 47 percent) were 
employed in the month preceding the survey than two years earlier. Employed mothers in the 
control groups had average earnings of approximately $6. 10 an hour, but many (between 1 1 and 
25 percent) were working at or below minimum wage. 

Considering all sources of income 13 , an estimated 69 (in Grand Rapids) to 77 percent (in 
Atlanta) of households in the control groups were living below the poverty line 14 at the two-year 
follow-up. Not surprisingly (given the population studied in the evaluation), these rates exceed 
the 17.4 percent of families with related children in the U.S. as a whole who were living in 
poverty in 1994. In addition, 1 1 percent in Riverside, 19 percent in Atlanta, and 21 percent in 
Grand Rapids, reported total net adjusted household incomes reflective of "deep poverty" (i.e., 
less than 50 percent of the poverty line). This compares to 7.2 percent of families in deep 
poverty nationwide (see Figure SR-5). 

In the absence of_ a JOBS, program, how were families faring in terms of non-economic 
aspects of family life? 

There were also indications of a range of risk on the non-economic aspects of family life 
among control group mothers. 

Depressive symptomatology . More than a quarter of the mothers in each site (from 29 
percent in Riverside to 37 percent in Atlanta and Grand Rapids) reported "many" depressive 
symptoms, suggestive of clinical levels of depression. The rates in Atlanta and Grand Rapids are 



'includes all earnings, AFDC, food stamps, WIC, SSI, social security, any unemployment insurance or 
workers’ compensation, refugees assistance, foster care payments, family or friends outside the household, 
estimated earned income tax credit, and is net of estimated child care expenses. 

l4 The 1994 poverty level for a single mother with two children was $1 1,940 and for a single mother with 
three children, $15,081. This poverty line for the appropriate household size was divided by 12. 
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Figure SR-5 

Percent of Families or Households Living in Poverty 

!■< 50% of poverty line ("deep poverty") □< 100% (in poverty) Dabove poverty 
100% n .I ii. , 




with Related Child Outcomes Study Control Groups 

Children < age 18 (% of households) 

in U.S. (1994) 



Sources: Current Population Survey - Annual Demographic Survey, March Supplement . [On-line] available: http://ferret.bls.census.gov 
Child Outcomes Study calculations by Child Trends. 



comparable to those reported by mothers in Atlanta in the months immediately following random 
assignment, when 42 percent of these mothers reported high levels of depressive symptoms 
(Moore et al., 1995). However, rates of "many" depressive symptoms are lower in the Child 
Outcomes Study sample compared to the more disadvantaged mothers enrolled in the New 
Chance Demonstration (all of whom were teenage mothers who had dropped out of school at the 
start of that evaluation); 52 percent of these mothers had high levels of depressive symptoms 
(Quint et al., 1997). Nonetheless, rates of depressive symptomatology suggestive of clinical 
depression among control group mothers in the Child Outcomes Study are higher than those 
typically found in community samples, which are closer to 20 percent for women (see review in 
Devins and Orme, 1985) (Figure SR-6). 

Parenting. Interviewers rated a nontrivial minority of control group mothers (between 8 
and 19 percent) as behaving in a harsh manner toward the focal child during the interview. 
Interviewers also rated control group mothers as moderately warm, on average, in interacting 
with their children, but there was a wide range in ratings of warmth, and ratings varied by site. 

On average, control group mothers did not rate themselves as particularly aggravated in the 
parenting role. However, about 1 0 percent of mothers in each site had scores indicating 
substantial aggravation and stress in the parenting role. 
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Figure SR-6 
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Moore, K. A., Zaslow, M. J., Corio, M. J„ Miller, S. & Magenheim, E. (1995). How well are they faring? AFDC families with preschool-aged children in Atlanta 
at the outset of the JOBS Evaluations . Washington, DC: U S. Department of Health and Human Services. 

Quint etal., (1997). 

Child Outcomes Study Calculations by Child Trends. 



Father involvement as reported bv mothers. We first examined the proportion of focal 
children living with their biological fathers. For those not living with their fathers, we note the 
proportion who had seen their fathers in the past 12 months. We also note the proportion of 
nonresidential fathers who had purchased clothing, toys or presents; had provided groceries; and 
who had served as a babysitter for the child. Few control group mothers (between 4 and 12 
percent) were living with the focal child’s biological father at the time of the follow-up survey, 
though over half of focal children who were not living with their biological fathers had seen them 
in the previous 12 months. In terms of other forms of support (buying clothes, toys, presents; 
providing groceries; occasional babysitting), focal children’s biological fathers provided, on 
average, less than one of the three kinds of support. 

Marital status and fertility. Whereas only 2 percent or fewer of control group mothers 
had been married at baseline, between 4 and 15 percent of control group mothers were married at 
the two-year follow-up. In addition, between 12 percent and 19 percent reported having had a 
baby between baseline and the two-year follow-up. 
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B. Program Impacts on Families 



Program impact findings for families in the full sample of the National Evaluation of 
Welfare-to-Work Strategies have been presented elsewhere (Hamilton et al., 1997; Freedman et 
al., 2000). Here we summarize findings specifically for the Child Outcomes Study sample. 
These findings help ascertain whether the JOBS programs brought about changes in economic 
and non-economic aspects of family life specifically in families that entered the evaluation with 
preschool-age children. It is important to determine if the presence of such young children 
impeded program participation or prevented families from making economic progress. It is also 
important to ascertain whether being assigned to a JOBS program when at least one child in the 
family was of preschool age resulted in changes in parenting behavior or the home environment, 
or in maternal psychological well-being. 

Did assignment to a JOBS_ program result in increased participation in program-related 
activities for motliers with vouns children? 

Regardless of program approach, the six JOBS programs studied resulted in greater 
participation in job search activities. The largest impacts on participation in job search activities 
were concentrated in labor force attachment programs. Mothers in each site’s labor force 
attachment program were about ten times as likely as those in the control groups to have ever 
participated (for at least one day) in job search activities since random assignment. Large 
impacts also occurred for mothers assigned to Riverside’s human capital development program: 
29 percent of these program mothers had participated in job search activities, compared to less 
than 5 percent of mothers in Riverside’s (human capital development) control group. 

Participation in basic education was higher for mothers assigned to a human capital 
development program than for control group mothers. The largest impact on participation in 
basic education activities (regardless of whether a degree was ultimately obtained) occurred for 
mothers assigned to Riverside’s human capital development program: over half of these program 
mothers had participated in basic education activities since random assignment, whereas only 16 
percent of control group mothers had done so. 

Corresponding to the participation mandate faced by program, but not control group, 
mothers, all six programs in the Child Outcomes Study resulted in sanctioning of program 
families who failed to comply with the participation mandate. Sanctioning rates were highest for 
mothers in the two Grand Rapids programs (27 percent for those in the human capital 
development program and 38 percent in the labor force attachment program). Mothers in 
Atlanta’s human capital development program were the least likely of the three human capital 
development programs to be sanctioned (15 percent as opposed to 27 percent in Grand Rapids 
and 24 percent in Riverside). Similarly, sanctioning rates were lower for mothers in Atlanta’s 
labor force attachment than in the other two labor force attachment programs (1 1 percent in 
Atlanta; 38 percent in Grand Rapids, and 14 percent in Riverside). 
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Did assignment to a JOBS program remit in impacts on targeted outcomes and/or on 
outcomes derivative of. targeted outcomes for families with young children ? 

Educational attainment . Mothers in the human capital development programs, but 
generally not those in the labor force attachment programs, progressed in terms of educational 
attainment. This is in keeping with program goals: labor force attachment programs generally 
did not aim to increase educational attainment, while human capital development programs did. 

Overall, mothers in human capital development programs in each site were more likely 
than controls to have received any educational credential since random assignment. By contrast, 
each site’s labor force attachment program tended not to improve mothers’ educational 
attainment (except in Atlanta, where mothers in the labor force attachment program were more 
likely than controls to have obtained a trade degree). Indeed, mothers assigned to Grand Rapids’ 
labor force attachment program were less likely than controls to have obtained a high school 
diploma or GED, trade certificate, or any degree since random assignment. 

Economic circumstances. Two years after random assignment, improvements in 
families’ economic circumstances occurred mainly for mothers assigned to labor force 
attachment programs and for mothers assigned to Riverside’s human capital development 
program. Impacts were largest and most numerous for the two programs in Riverside. 

Program mothers in Atlanta witnessed relatively few economic impacts. From among 
multiple economic outcome measures, impacts were found on only a few in this site. As 
intended, both programs led to a reduction in AFDC receipt at the two-year follow-up. In 
addition, those in Atlanta’s human capital development program were less likely to report 
earning the minimum wage or less and had somewhat greater earnings from their current jobs, 
and those in Atlanta’s labor force attachment program were more likely to be employed, and less 
likely to be in deep poverty in the month prior to the two-year follow-up. Neither of Atlanta’s 
JOBS programs had a statistically significant impact on total net adjusted household income at 
the two-year point. 

The only statistically significant unfavorable economic impacts of any of the six JOBS 
programs in this study occurred for mothers in Grand Rapids’ JOBS programs. Both JOBS 
programs in Grand Rapids’ reduced the proportion of families at or above the poverty line. 
Mothers assigned to Grand Rapids’ human capital development program were less likely to be 
employed 40 or more hours perxweek, and results for this program show an increase in the 
proportion of mothers working for less than the minimum wage. The only favorable economic 
impact of Grand Rapids’s labor force attachment program was on the likelihood that program 
mothers were employed at some point since random assignment. It is worth noting that those in 
Grand Rapids’ labor force attachment program had greater employment rates in the month prior 
to the two-year follow-up than those in any of the other sites’ JOBS programs, but because 
control group mothers in Grand Rapids also had relatively high rates of employment (compared 
to control group mothers in the other sites), there was no net impact of Grand Rapids’ labor force 
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attachment program on current employment. Neither of Grand Rapids’ JOBS programs had a 
statistically significant impact on total net adjusted household income at the two-year point. 

Economic impacts in Riverside were more widespread. Mothers assigned to either of 
Riverside’s JOBS programs were more likely than controls to have been employed at some point 
since random assignment, were more likely to have been employed in the month prior to the two- 
year follow-up, had more hours of employment, higher wages, and greater earnings. (Riverside’s 
JOBS programs were the only ones in this study to have favorable impacts on hourly wage and 
hours employed.) In addition, mothers enrolled in Riverside’s labor force attachment program 
were less likely to be receiving AFDC, and mothers in Riverside’s human capital development 
program were more likely than their control group counterparts to have incomes above the 
poverty line. These impacts also occurred for many lower-risk and higher-risk subgroups. In 
addition, despite the absence of impacts on total income at the aggregate level, Riverside’s 
human capital development program resulted in higher total household income for one lower-risk 
and two higher-risk subgroups of mothers. However, for another lower-risk subgroup, this 
program resulted in lower earnings. Also, despite the absence of impacts on deep poverty at the 
aggregate level, Riverside’s labor force attachment program increased deep poverty for some 
higher-risk families. Nevertheless, employment and economic impacts of Riverside’s JOBS 
programs were generally favorable, which is consistent with this site’s focus and experience 
implementing work-oriented programs. 

Employment-related child care. Program mothers (except those in Grand Rapids’ 
human capital development program) were more likely than controls to be using child care for 
any of their children while they were employed at their current or most recent job. The largest 
impact on the use of employment-related child care for any of their children occurred for 
Riverside’s labor force attachment program, where program mothers experienced the largest 
employment impacts (and where control group mothers reported relatively infrequent 
employment-related child care). Relatively large impacts also occurred in Riverside’s human 
capital development program and in Grand Rapids’ labor force attachment program, where 
program mothers were more likely than controls to have been employed at some point since 
random assignment. The small impacts in Atlanta likely reflect, in part, the absence of large 
employment impacts in this site. 

Impacts on the current use of employment-related child care for the focal child were more 
limited and occurred only in Riverside: Only Riverside’s JOBS programs increased the overall 
likelihood that mothers were both employed and using any non-maternal care for the focal child 
in the month prior to the two-year survey. Regardless of program approach, program mothers in 
this site reported more employment-related hours of child care for the focal child in the prior 
month, were more likely than controls to be employed and using informal care for the focal child, 
and to be employed and using care for the focal child during irregular hours or with a varying 
schedule. These findings are consistent with the focus in Riverside -- likely experienced more 
intensively by program than control group mothers - on helping mothers arrange low cost, 
primarily informal, child care, which provides greater flexibility as to hours of employment. 
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Didjtssigmnent to a JOBS program affect further . non-tarseted outcomes for families? 

There was less evidence of impacts on aspects of family life not targeted by JOBS 
welfare -to-work programs, yet such impacts did occur. When these impacts occurred, however,, 
there were indications (discussed below) that they could be quite important to child outcomes. 

Maternal psychological well-being. Specific aspects of maternal psychological well- 
being were affected in different ways by JOBS welfare-to-work programs, and impacts did 
emerge, especially for subgroups of mothers. 

Assignment to a JOBS welfare-to-work program often led to increased feelings of time 
stress. Perhaps more worrisome, enrollment in a JOBS program increased depressive 
symptomatology, in the aggregate, for mothers assigned to Grand Rapids’ labor force attachment 
program, as well as for a few lower-risk subgroups exposed to Grand Rapids’ or Riverside’s 
human capital development program, and a lower-risk subgroup assigned to Atlanta’s labor force 
attachment program. Interestingly, Atlanta’s labor force attachment program increased the 
number of depressive symptoms among mothers reporting relatively many psychological 
problems at baseline, and decreased the number of depressive symptoms among mothers 
reporting no or relatively few psychological problems at baseline. This pair of offsetting impacts 
indicate that this JOBS program had completely opposite impacts (which resulted in an absence 
of impacts in the aggregate) depending on the initial level of mothers’ depressive symptoms. 

At the same time, findings indicate that, despite an absence of impacts at the aggregate 
level, four of the six JOBS programs led to greater feelings of control over one’s life for certain 
(especially lower-risk) subgroups of mothers. This was especially true for Riverside’s labor 
force attachment program, in which a majority of lower-risk subgroups examined had increases 
on the measure of subjective sense of control. 

Parenting . In the aggregate, Atlanta’s human capital development program led to greater 
warmth (as reported by the mother) and to higher scores on a summary measure of "favorable" 
parenting. Additional impacts on parenting emerged for a few subgroups, with Atlanta’s human 
capital development program having both favorable and unfavorable impacts in both lower-risk 
and higher-risk subgroups. In the aggregate, Atlanta’s labor force attachment program led to less 
harsh discipline, greater maternal warmth (as reported by the interviewer), greater verbal 
interactions with the focal child, and higher scores on a summary measure of "favorable" 
parenting. The additional impacts that emerged in a few subgroups were uniformly favorable in 
lower-risk subgroups and were both favorable and unfavorable in higher-risk subgroups. 

There was only one aggregate impact on parenting in Grand Rapids: Mothers assigned to 
Grand Rapids’ labor force attachment program reported less warmth toward the focal child than 
did control group mothers. Additional parenting impacts emerged in subgroups, and most were 
unfavorable — regardless of risk level. 
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In the aggregate, there were no impacts of either of Riverside’s JOBS programs on any 
measure of parenting. Nevertheless, a few impacts emerged at the subgroup level, with both 
favorable and unfavorable impacts of each program in lower-risk subgroups, but only favorable 
impacts of each program in higher-risk subgroups. 

Father involvement as reported by mothers. Mothers in each of Grand Rapids JOBS 
programs reported receiving more forms of support from the focal child’s biological father, such 
as purchasing groceries, toys, clothes, and babysitting, compared to control group mothers. This 
difference, while statistically significant, was small. Further, such support was reported to be 
occurring at low levels: program mothers in Grand Rapids still reported less than one kind of 
support, on average. Riverside’s labor force attachment program decreased the likelihood that 
the focal child’s biological father lived with the focal child at the two-year follow-up. Additional 
impacts emerged when subgroups were considered; notably, each of Atlanta’s JOBS programs 
decreased the likelihood that the focal child’s father lived with him or her in four higher-risk 
subgroups. In Grand Rapids, all impacts on father involvement in subgroups were favorable. By 
contrast, in Riverside, all impacts on father involvement in subgroups were unfavorable. 

Marriage and fertility . In the aggregate, none of the six welfare-to-work programs had 
an impact on mothers’ fertility, or their marital and cohabitation status, by the two-year follow- 
up, though a few scattered impacts emerged for five of the six JOBS programs in both lower- and 
higher-risk subgroups. 

VII. Findings: 

Linking Impacts on Children to Impacts on Families 

Through what pathways do children appear to have been affected by their mothers’ 
assignment to a JOBS welfare-to-work program? In this section, we summarize results from 
statistical analyses that identify which program impacts on targeted and non-targeted outcomes 
appear to underlie selected impacts on children. The analyses of program impacts on children 
summarized above are experimental in nature. Experimental analyses provide strong causal 
evidence regarding the existence of program impacts on children; however, they cannot address 
the question of how these impacts came about. Analyses seeking to identify the pathways 
through which a program had its impacts are necessarily non-experimental. Because mediational 
analyses are non-experimental, they do not allow firm causal inferences to be made regarding the 
pathways through which these impacts came about. 15 In addition, the specific mediational 



l5 Notably, moving from experimental to non-experimental analyses creates some potential difficulties with 
respect to selection bias. For instance, because JOBS did not randomly assign mothers to various types of child 
care, mothers’ child care choices reflect, to some degree, their preferences, motivations, and other characteristics. 

As a hypothetical illustration, if mothers with problem behavior children are disproportionately more likely than 
mothers without problem behavior children to seek quality child care in hopes of curbing the problem behavior -- 
and assuming that quality child care truly leads, causally, to better behavioral outcomes in children — then the 
observed statistical "effect" of child care on children’s behavioral outcomes will underestimate the true positive 
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analyses conducted in this study rely on information about mediators and child outcomes 
measured contemporaneously and used a relatively modest statistical approach in modeling 
pathways; as such, results should be considered preliminary. 

To what extent were program impacts on measures of focal children 's cognitive school 
readiness, extenudizing. behavior problems , and general health explained by_ program impacts 
on mothers (Mu OH targeted and non = targeted outcomes [? 

In a modest attempt to understand the pathways through which specific welfare-to-work 
programs may affect children’s developmental outcomes, five of the aggregate impacts on focal 
children in the Child Outcomes Study sample were examined in more detail through mediational 
analyses. These five impacts were selected because they illustrate the general pattern of findings 
at the aggregate level of favorable cognitive, unfavorable health, and mixed behavioral impacts. 
In particular, we examine: 

A favorable impact on a measure of cognitive development : 

(1) the favorable impact of Atlanta’s labor force attachment program on focal children’s 
mean cognitive school readiness scores; 

Two unfavorable impacts on measures of children’s health : 

(2) the unfavorable impact of Riverside’s human capital development program on ratings 
of the focal child’s overall health; 

(3) the unfavorable impact of Riverside’s labor force attachment program on ratings of 
focal children’s overall health; 

A favorable and an unfavorable impact on behavioral and emotional adjustment: 

(4) the favorable inipact of Atlanta’s labor force attachment program on focal children’s 
reported externalizing behavior problems; and 



effect of child care. The mediational analyses reported on here and in the full report (as well as the experimental 
impact analyses that preceded them) did not control for children’s prior developmental and behavioral outcomes; 
however, we did control for numerous other variables representing prior characteristics of the child, mother, and 
family at baseline that may serve as selection factors. (For a complete list, see the third footnote of Chapter 10 of 
the full report.) The availability of this number and range of baseline variables is a great asset of this dataset, and 
previous research using these data indicate that many of these baseline variables do, to some extent, capture 
selection into employment (Zaslow, McGroder, Cave, and Mariner, 1999), selection into child care (Zaslow, 
Oldham, Magenheim, and Moore, 1998), and selection into parenting pattern (McGroder, 2000). Thus, we are 
likely to have controlled for many, though not all, possible selection effects. 
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(5) the unfavorable impact of Grand Rapids’ labor force attachment program on focal 
children’s reported externalizing behavior problems. 

Mediational analyses are statistical analyses that attempt to identify one or more variables 
that appear to explain, statistically, the relationship between two other variables (see Baron and 
Kenny, 1986). In the present study, targeted and non-targeted outcomes that were affected by a 
given JOBS program (i.e., there was an experimental impact on the outcome) were examined as 
possible "mediators" of the program impact on children. The mediational analyses ask whether 
the statistical significance of the association between the JOBS program and a child outcome (the 
significant impact) is diminished or eliminated when the mediating variable, or set of mediating 
variables, is taken into account. When this happens, there is evidence that the mediator is a 
conduit through which the child impact is coming about. In these analyses, we find that some of 
the child impacts are "fully mediated" (the statistical significance of the impact on a child 
outcome is eliminated when considering the role of the mediators), while in other instances there 
was only "partial mediation" (the statistical significance of the impact on the child outcome is 
diminished but not eliminated when considering the mediators). 

It is also possible that a mediator operates to increase (instead of decrease) the statistical 
significance (and, thus, the magnitude of the experimental impact) of a given JOBS program on a 
child outcome. In this case, the mediator is not one that helps to explain the experimental impact 
on children. Instead, this mediator is operating in an opposing direction, indicating that the 
program impact on the given child outcome would have been even more pronounced if not for 
this variable’s buffering, or offsetting, effect. 

Mediational analyses first examined for which of the 60 targeted and non-targeted adult 
and family outcomes studied in the COS there were statistically significant impacts of any of the 
six JOBS programs studied (see Chapter 9, Table 9.1). Next, for each particular analysis (i.e., 
focusing on one of the selected impacts on children noted above), the set of variables that was 
included in each specific model was then narrowed to those adult and family outcomes on which 
there were statistically significant impacts in the program we were focusing on. Of all the adult 
and family outcomes that emerged as mediators in any of these five analyses (see Chapter 10, 
Table 10.1), three sets of adult variables consistently emerged as important in explaining child 
impact findings: variables related to the mothers’ employment, to the mothers’ parenting, and to 
maternal psychological well-being. 

Cognitive development. The favorable impact of Atlanta’s labor force attachment 
program on focal children’s mean cognitive school readiness scores appears to be related 
to program mothers’ greater employment and improvements in parenting. 

Health . The unfavorable impact of Riverside’s human capital development program on 
mothers’ ratings of focal children’s overall health appears to .be related to program 
mothers’ increased feeling of time stress. However, program impacts would have been 
even more unfavorable in the ratings of children’s health if program mothers were not 



also more likely to have been sanctioned, to have participated in basic education, and 
obtained a high school diploma or GED. (Perhaps these factors— being sanctioned, 
participating in basic education, and Obtaining a high school diploma or GED— go 
together, indicating a program in which there was both more encouragement and pressure 
from case workers to participate, and indeed resulting in more participation and 
educational progress). 16 

The unfavorable impact of Riverside’s labor force attachment program on ratings of focal 
children’s overall health appears to be related to decreases in AFDC receipt and increases 
in mothers’ work hours, although the picture remains somewhat unclear. 17 One 
possibility that would require further examination is that the loss of cash welfare benefits 
led to a loss of Medicaid coverage which, in turn, if coverage was not replaced by 
employer-provided coverage or other types of coverage, led to a decline in children’s 
overall health ratings. ; 

Behavioral and emotional adjustment . The favorable impact of Atlanta’s labor force 
attachment program on focal children’s reported externalizing behavior problems appears 
to be related to the program’s favorable impact on parenting, despite also increasing 
mothers’ reports of time stress and perceptions of feeling "pushed" by the welfare office. 

The unfavorable impact of Grand Rapids’ labor force attachment program on focal 
children’s reported externalizing behavior problems appears to be related to the 
program’s unfavorable impacts on maternal depressive symptomatology and parenting. 

What have we learned about possible pathways through which various welfare-to- work 
programs can affect children ? 

Several themes emerge from these results. First, the mechanisms through which children 
can be affected by a given welfare-to-work program include both targeted outcomes (e.g., 
employment, AFDC receipt) as well as non-targeted outcomes (maternal psychological well- 
being and parenting). In particular, this study highlights the role played by intervening 
mechanisms more proximal to the child in the home environment — namely, maternal 
psychological well-being and parenting. Aspects of family life targeted by welfare-to-work 
programs emerged less consistently as mediators of the five child impacts selected for 
mediational analyses. It may be, however, that these targeted outcomes do play a role inasmuch 



l6 Future work may attempt to examine in greater detail how such variables are linked. 

17 The analyses for this particular child outcome indicate a statistically significant role for both AFDC 
receipt and mother’s work hours in models considering these variables separately (along with covariates), but not in 
a combined model. This appears to be due to multicollinearity of the mediating variables. For all other results, the 
findings summarized held in a final combined model as well as in models considering each mediating variable 
separately. 
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as program impacts on these outcomes activate processes more proximal to the child. For 
example, changes in employment or earnings may not, in and of themselves, consistently lead to 
program impacts on children’s behavior problems unless these lead to changes in parenting or 
depressive symptomatology. Testing such multi-step hypotheses with statistical methods that 
can model such hypotheses is warranted as a next step. 

It is worth noting that, because none of the six JOBS programs studied here had aggregate 
impacts on total household income, this impact could not serve as a pathway through which any 
of the aggregate impacts on children (not just the five impacts selected for further study) came 
about. 



In addition, it is also noteworthy that — despite pervasive program impacts on child care 
and more circumscribed program impacts on health insurance coverage - these impacts did not 
help to explain either the favorable or the unfavorable child impacts examined here. However, 
this does not mean that child care and health insurance coverage are not important to children’s 
outcomes - only that these variables were not found to explain the particular impacts examined 
here. The reader should keep in mind that we examined the relation between mediators and child 
outcomes in an experimental context, that is, only in cases where there was an experimental 
impact both on a mediator and on a child outcome. This addresses one specific type of relation 
between a mediating variable and a child outcome, namely, whether a given mediating variable — 
such as child care — plays a role in helping to explain a particular child impact finding. 

However, there are likely to be associations between child care and child outcomes in this sample 
even where there were no program impacts on child care and/or no impacts on children. Indeed, 
findings from the experimental literature indicate that quality preschool experiences can have 
lasting benefits for low-income children (Barnett, 1995), and findings from the non-experimental 
literature show that formal child care arrangements bode well for low-income children’s 
developmental outcomes (Zaslow, McGroder, Moore, and LeMenestrel, 1999; Zaslow, Oldham, 
Magenheim, and Moore, 1998). For example, a recent set of analyses shows that among control 
group families in the Riverside site of the Child Outcomes Study, five- to seven-year-old 
children identified to be at risk in terms of cognitive school readiness (because they were two or 
more years behind in basic concepts they should have mastered by school entry) were less likely 
to be currently enrolled in a formal child care arrangement (Zaslow, McGroder, Moore, and 
LeMenestrel, 1999). In short, because we restricted our focus to instances in which there was an 
impact on both a child outcome measure and a mediator, our analyses do not address the broader 
question of whether these mediators are important to children’s outcomes, in general. 

A second theme relates to the size of the mediating effects. Mediational analyses show 
that even relatively "small" 18 impacts on targeted outcomes (e.g., the favorable impact of 



18 As we have noted, researchers in the behavioral sciences often rely on Cohen’s (1988) characterization of 
effect sizes (in standard deviation units) of .20 as "small," .50 as "medium," and .80 as "large." Cohen (1988) 
acknowledges that this characterization is somewhat arbitrary and "is recommended for use only when no better 
basis for estimating the effect size index is available" (p. 25). For example, when the accumulation of empirical 



Atlanta’s labor force attachment program on mothers’ current employment) and non-targeted 
outcomes (e.g., the favorable impact of Atlanta’s labor force attachment program on measures of 
parenting; the unfavorable impact of Grand Rapids’ labor force attachment program on a 
measure of parenting) can help to explain relatively small program impacts on children. Thus, 
program impacts on mothers need not be "large" to translate into impacts on children. 

A third theme to emerge from these findings is that some welfare-to-work programs have 
effects in opposing directions on aspects of family life that are important to children. For 
example, Atlanta’s labor force attachment program increased mothers’ feelings of time stress 
(which was associated with more frequent externalizing behavior problems) but, at the same 
time, this program also led to improvements in parenting (which was associated with less 
frequent externalizing behavior problems). The net effect of these opposing influences for focal 
children’s externalizing behavior problems was favorable in this program. 

These findings suggest that, in modeling pathways through which children can be 
affected by welfare-to-work programs, it is imperative to consider both positive and negative 
pathways, regardless of whether one is trying to explain a favorable child impact or an 
unfavorable child impact. Influences of mediating variables going in contrasting directions may 
help explain the small number and size of significant impacts on child outcomes: counteracting 
influences may, at times, result in no or little "net" influence on children. Understanding such 
influences is central to strengthening pathways that yield favorable impacts on children. 

As noted above, because the present analyses rely on information about mediators and 
child outcomes measured contemporaneously, results should be considered preliminary. 
Additional waves of data from the five-year follow-up, as well as statistical methods that more 
effectively partition effects among multiple mediators, more completely control for selection 
effects, and allow alternative models to be tested explicitly (e.g., structural equation modeling), 
are needed to provide more definitive answers regarding the multiple pathways through which 
program impacts on children came about. Nevertheless, given the limited state of knowledge on 
the pathways through which welfare-to-work programs can affect children and families, it is 
important to begin to address this issue, statistically , even with contemporaneous measures of 
mediators and child outcomes and using a relatively modest statistical approach. We expect that 
the mediational results reported here begin to shed light on this important topic, and will serve as 
the basis for model-building and testing explicit hypotheses in the future, especially when five- 
year data become available. 



research demonstrates that a given effect size is predictive of a "meaningful" difference in an outcome generally 
agreed to be "important," this effect size may well be considered "large" regardless of its absolute size. 
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VIII. Summary and Implications 



Overall, the results described here indicate that the welfare-to-work programs 
implemented as part of the JOBS Program did have significant impacts on children’s 
development, but these impacts were not widespread and were generally small . There were not 
strong indications of unfavorable impacts on children when their mothers were assigned to 
participate in a JOBS program, and indeed it is important to note that the significant impacts 
included favorable impacts on children. 

In the aggregate, program impacts on children were not found on many outcomes, and 
those impacts that were found tended to be small. In addition, the direction of impacts differed 
for the three aspects of development examined: There were favorable impacts particularly in the 
area of cognitive development, and particularly in the Atlanta site, while there were unfavorable 
(albeit generally small) impacts in the area of health, specifically in the Riverside site. Findings 
in the area of children’s behavioral and emotional adjustment encompassed both favorable and 
unfavorable impacts. Only a single finding at the aggregate level (an unfavorable impact on a 
rating of child health in Riverside’s labor force attachment program ) was of sufficient magnitude 
to be called "policy relevant." 

In terms of subgroup impacts, we acknowledged that children from families at higher risk 
could show a pattern of unfavorable or of favorable child impacts. However, we hypothesized 
that children from higher-risk families might experience disproportionately unfavorable impacts 
resulting from their mothers’ assignment to a JOBS program. 

As for impacts at the aggregate level, findings at the subgroup level were few and tended 
generally to be small. The subgroup impact findings identified a pattern of unfavorable impacts, 
with a number of these large enough to be considered policy relevant, for children from lower- 
nsk families. In particular, such a pattern occurred for children from lower-risk families in three 
programs: Grand Rapids’ labor force attachment program (in which unfavorable and policy 
relevant impacts were found on behavioral and health outcomes); Riverside’s human capital 
development program (again with unfavorable and policy relevant impacts in the areas of child 
behavior and health); and in Riverside’s labor force attachment program (with unfavorable and 
policy relevant impacts in all three aspects of development). It is also worth noting that some of 
these unfavorable and policy relevant impacts may be considered "moderate" to "large" in terms 
of effect sizes. The fact that the unfavorable and policy relevant impacts occurred in a 
concentrated manner (for lower-risk families in particular programs), and that some of these were 
"moderate" to "large" in magnitude, suffices to indicate that we cannot entirely dismiss the 
possibility of unfavorable impacts. It will be important to continue to track findings for children 
in lower-risk families assigned to these programs in the five-year follow-up of the Child 
Outcomes Study. 

At the same time, there was a pattern of favorable program impacts for children in higher- 
risk families whose mothers had been assigned to human capital development programs or to 
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Atlanta’s labor force attachment program. 19 These favorable impacts were generally small, 
however, with none of sufficient magnitude to be called policy relevant. Again this pattern needs 
to be followed over time. Interestingly, all four of these programs also had favorable impacts on 
mothers’ educational attainment (e.g., receipt of a high school diploma or GED; receipt of a trade 
degree). We need to determine if, in the longer-term, programs that improve mothers’ 
educational outcomes continue to have beneficial implications for children from higher-risk 
families. 

In sum, most children were not adversely affected — and in some cases, children were 
helped — by their mothers’ participation in a JOBS program. Yet for delimited subgroups 
(particularly lower-risk families in both of Riverside’s programs and in Grand Rapid’s labor 
force attachment program) there is reason for vigilance. Further follow-up of the children (five 
years after enrollment) will be important. 

Given these findings, we must consider it a possibility that the welfare-to-work programs 
put in place by the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 
and implemented in the context of other policy changes, such as increased child care funding, 
increased earnings disregards, and Medicaid and SCHIP eligibility expansions, can have impacts 
on children’s development. It is indeed possible that the different obligations and opportunities 
of the new welfare policy will result in stronger or more pervasive impacts on children than we 
have found here. Yet we should not begin with an assumption that the newer programs will have 
uniformly unfavorable or favorable impacts on children. Rather, the present findings suggest 
that outcomes for children are likely to differ by domain of development, by the specific features 
of local programs, and by characteristics of the families. The match between type of program 
and whether the family entered the program at higher or lower risk may continue to prove 
important. Regarding children, then, the present findings underscore the importance of 
continuing to monitor the magnitude and direction of program impacts on children’s 
developmental outcomes in the new policy context. 

We have learned that welfare-to-work programs implemented under JOBS had the 
potential to affect multiple aspects of family life, including aspects of family life not explicitly 
targeted by the programs. Non-experimental statistical analyses indicate that impacts on children 
sometimes reflected the simultaneous influence of both favorable and unfavorable program 
effects on families. Further, we have learned that child impacts reflect multiple program 
influences on families. Pathways through which child impacts appear to have come about in the 
present study included JOBS programs’ impacts on outcomes explicitly targeted by the programs 
(such as employment and receipt of AFDC) and especially, impacts on further, non-targeted 
aspects of family life (such as parenting behavior and maternal psychological well-being). 



l9 The only exceptions relate to unfavorable health impacts of Riverside’s human capital development 
program, a single unfavorable health impact of Atlanta’s labor force attachment program, and a single unfavorable 
behavioral impact of each of Atlanta’s JOBS programs. 
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Thus far it appears that the new welfare-to-work policies implemented in the states 
following the passage of PRWORA, combined with a robust economy and other policies, have 
contributed to recent changes in employment rates and on receipt of public assistance (Moffitt, 
1999). The fact that impacts on children in the present study were linked to such targeted 
outcomes underscores the possibility that current policies may also be having effects on children. 
The present set of findings point to the importance of examining not only the effects of current 
policies on employment, receipt of public assistance, earnings and income, but also on such 
aspects of family life as parenting behavior and mothers’ psychological well-being. 

In sum, we have learned that welfare-to-work programs do indeed have the potential to 
affect children, both favorably and unfavorably. It is important to identify whether, to what 
extent, and how, impacts on children are occurring in the present policy context. Welfare 
policies originated in a concern for the well-being of children. Effects on children should 
continue to be monitored and considered in policy decisions. 
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